研究园简易版 · 经典书籍 · 期刊文章 · 有原文 · 完整版(可多选筛选、搜索) · 首页
2020 年的期刊文章 · 第 31 页
共 1467 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2020 到 2020。
按期刊、按主题看 →
JMLR Machine Learning Networks & Graphs · 发表于 2020
Enguerrand Horel, Kay Giesecke
摘要
We develop a pivotal test to assess the statistical significance of the feature variables in a single-layer feed-forward neural network regression model. We propose a gradient-based test statistic and study its asymptotics using non-parametric techniques. Under technical conditions, the limiting distribution is given by a mixture of chi-square distributions. The tests enable one to discern the impact of individual variables on the prediction of a neural network. The test statistic can be used to rank variables according to their influence. Simulation results illustrate the computational efficiency and the performance of the test. An empirical application to house price valuation highlights the behavior of the test using actual data.
JMLR High-Dimensional Networks & Graphs Spatial & Time Series · 发表于 2020
Hossein Keshavarz, George Michaildiis, Yves Atchade
摘要
High dimensional piecewise stationary graphical models represent a versatile class for modelling time varying networks arising in diverse application areas, including biology, economics, and social sciences. There has been recent work in offline detection and estimation of regime changes in the topology of sparse graphical models. However, the online setting remains largely unexplored, despite its high relevance to applications in sensor networks and other engineering monitoring systems, as well as financial markets. To that end, this work introduces a novel scalable online algorithm for detecting an unknown number of abrupt changes in the inverse covariance matrix of sparse Gaussian graphical models with small delay. The proposed algorithm is based upon monitoring the conditional log-likelihood of all nodes in the network and can be extended to a large class of continuous and discrete graphical models. We also investigate asymptotic properties of our procedure under certain mild regularity conditions on the graph size, sparsity level, number of samples, and pre- and post-changes in the topology of the network. Numerical works on both synthetic and real data illustrate the good performance of the proposed methodology both in terms of computational and statistical efficiency across numerous experimental settings.
Econometrica Other · 发表于 2020
Laura Doval, Jeffrey C. Ely
摘要
We study games of incomplete information as both the information structure and the extensive form vary. An analyst may know the payoff‐relevant data but not the players' private information, nor the extensive form that governs their play. Alternatively, a designer may be able to build a mechanism from these ingredients. We characterize all outcomes that can arise in an equilibrium of some extensive form with some information structure. We show how to specialize our main concept to capture the additional restrictions implied by extensive‐form refinements.
Biometrics Clinical Trials Causal Inference · 发表于 2020
Yasuhiro Hagiwara, Tomohiro Shinozaki, Hirofumi Mukai, Yutaka Matsuyama
摘要
Subsequent treatments can result in a difficulty in interpretation of the overall survival results in confirmatory oncology clinical trials. To complement the intention‐to‐treat (ITT) analysis affected by subsequent treatment patterns unintentional in the trial protocol, several causal methods targeting the per‐protocol effect have been proposed. When two or more types of subsequent treatments are allowed in the trial protocol, however, these methods cannot answer clinical questions such as how sensitive the ITT analysis result is to higher or lower proportions of each subsequent treatment allowed in the trial protocol than observed, and to what extent ITT analysis result is generalizable to subsequent treatment patterns other than observed one. To answer these clinical questions, we propose a sensitivity analysis method for subsequent treatments using the inverse probability of treatment weighting method for stochastic dynamic treatment regimes (DTRs). We formulate oncology clinical trials with subsequent treatments as two‐stage designs in which initial treatments are randomized, but subsequent treatments are observational. In this formulation, we use stochastic DTRs to simulate specific proportions of each subsequent treatment and compare an initial experimental treatment with an initial control treatment under various proportions of each subsequent treatment. We applied our proposed method to a motivating randomized noninferiority trial for metastatic breast cancer. Simulation results are also reported to show the usefulness of the proposed method.
JMLR Spatial & Time Series · 发表于 2020
Ganggang Xu, Ming Wang, Jiangze Bian, Hui Huang, Timothy R. Burch, Sandro C. Andrade, Jingfei Zhang, Yongtao Guan
摘要
We propose a general framework of using a multi-level log-Gaussian Cox process to model repeatedly observed point processes with complex structures; such type of data has become increasingly available in various areas including medical research, social sciences, economics, and finance due to technological advances. A novel nonparametric approach is developed to efficiently and consistently estimate the covariance functions of the latent Gaussian processes at all levels. To predict the functional principal component scores, we propose a consistent estimation procedure by maximizing the conditional likelihood of super-positions of point processes. We further extend our procedure to the bivariate point process case in which potential correlations between the processes can be assessed. Asymptotic properties of the proposed estimators are investigated, and the effectiveness of our procedures is illustrated through a simulation study and an application to a stock trading dataset.
JMLR Optimization & Learning Theory · 发表于 2020
Fan Ma, Deyu Meng, Xuanyi Dong, Yi Yang
摘要
Co-training is a well-known semi-supervised learning approach which trains classifiers on two or more different views and exchanges pseudo labels of unlabeled instances in an iterative way. During the co-training process, pseudo labels of unlabeled instances are very likely to be false especially in the initial training, while the standard co-training algorithm adopts a 'draw without replacement' strategy and does not remove these wrongly labeled instances from training stages. Besides, most of the traditional co-training approaches are implemented for two-view cases, and their extensions in multi-view scenarios are not intuitive. These issues not only degenerate their performance as well as available application range but also hamper their fundamental theory. Moreover, there is no optimization model to explain the objective a co-training process manages to optimize. To address these issues, in this study we design a unified self-paced multi-view co-training (SPamCo) framework which draws unlabeled instances with replacement. Two specified co-regularization terms are formulated to develop different strategies for selecting pseudo-labeled instances during training. Both forms share the same optimization strategy which is consistent with the iteration process in co-training and can be naturally extended to multi-view scenarios. A distributed optimization strategy is also introduced to train the classifier of each view in parallel to further improve the efficiency of the algorithm. Furthermore, the SPamCo algorithm is proved to be PAC learnable, supporting its theoretical soundness. Experiments conducted on synthetic, text categorization, person re-identification, image recognition and object detection data sets substantiate the superiority of the proposed method.
Econometrica Other · 发表于 2020
Bo E. Honoré, Luojia Hu
摘要
It is well understood that classical sample selection models are not semiparametrically identified without exclusion restrictions. Lee (2009) developed bounds for the parameters in a model that nests the semiparametric sample selection model. These bounds can be wide. In this paper, we investigate bounds that impose the full structure of a sample selection model with errors that are independent of the explanatory variables but have unknown distribution. The additional structure can significantly reduce the identified set for the parameters of interest. Specifically, we construct the identified set for the parameter vector of interest. It is a one‐dimensional line segment in the parameter space, and we demonstrate that this line segment can be short in practice. We show that the identified set is sharp when the model is correct and empty when there exist no parameter values that make the sample selection model consistent with the data. We also provide non‐sharp bounds under the assumption that the model is correct. These are easier to compute and associated with lower statistical uncertainty than the sharp bounds. Throughout the paper, we illustrate our approach by estimating a standard sample selection model for wages.
Econometrica Other · 发表于 2020
Jacob Moscona, Nathan Nunn, James A. Robinson
摘要
We test the longstanding hypothesis that ethnic groups organized around “segmentary lineages” are more prone to conflict. Ethnographic accounts suggest that in such societies, which are characterized by strong allegiances to distant relatives, individuals are obligated to come to the aid of fellow lineage members when they become involved in conflicts. As a consequence, small disagreements often escalate into larger‐scale conflicts involving many individuals. We test for a link between segmentary lineage organization and conflict across ethnic groups in sub‐Saharan Africa. Using a number of estimation strategies, including a regression discontinuity design at ethnic boundaries, we find that segmentary lineage societies experience more conflicts, and particularly ones that are retaliatory, long in duration, and large in scale.
JMLR Networks & Graphs Latent Variables & Mixtures Meta-Analysis & Data Integration · 发表于 2020
Thomas Bonald, Nathan de Lara, Quentin Lutz, Bertrand Charpentier
摘要
Scikit-network is a Python package inspired by scikit-learn for the analysis of large graphs. Graphs are represented by their adjacency matrix in the sparse CSR format of SciPy. The package provides state-of-the-art algorithms for ranking, clustering, classifying, embedding and visualizing the nodes of a graph. High performance is achieved through a mix of fast matrix-vector products (using SciPy), compiled code (using Cython) and parallel processing. The package is distributed under the BSD license, with dependencies limited to NumPy and SciPy. It is compatible with Python 3.6 and newer. Source code, documentation and installation instructions are available online.
JMLR Bayesian Methods High-Dimensional · 发表于 2020
James Johndrow, Paulo Orenstein, Anirban Bhattacharya
摘要
The horseshoe prior is frequently employed in Bayesian analysis of high-dimensional models, and has been shown to achieve minimax optimal risk properties when the truth is sparse. While optimization-based algorithms for the extremely popular Lasso and elastic net procedures can scale to dimension in the hundreds of thousands, algorithms for the horseshoe that use Markov chain Monte Carlo (MCMC) for computation are limited to problems an order of magnitude smaller. This is due to high computational cost per step and growth of the variance of time-averaging estimators as a function of dimension. We propose two new MCMC algorithms for computation in these models that have significantly improved performance compared to existing alternatives. One of the algorithms also approximates an expensive matrix product to give orders of magnitude speedup in high-dimensional applications. We prove guarantees for the accuracy of the approximate algorithm, and show that gradually decreasing the approximation error as the chain extends results in an exact algorithm. The scalability of the algorithm is illustrated in simulations with problem size as large as $N=5,000$ observations and $p=50,000$ predictors, and an application to a genome-wide association study with $N=2,267$ and $p=98,385$. The empirical results also show that the new algorithm yields estimates with lower mean squared error, intervals with better coverage, and elucidates features of the posterior that were often missed by previous algorithms in high dimensions, including bimodality of posterior marginals indicating uncertainty about which covariates belong in the model.
Econometrica Other · 发表于 2020
Lorenz Hartmann
摘要
Savage (1954) provided the first axiomatic characterization of expected utility without relying on any given probabilities or utilities. It is the most famous preference axiomatization existing. This note shows that Savage's axiom P3 is implied by the other axioms, which reveals its redundancy. It is remarkable that this was not noticed before as Savage's axiomatization has been studied and taught by hundreds of researchers for more than six decades.
Econometrica Other · 发表于 2020
Alberto Abadie, Susan Athey, Guido W. Imbens, Jeffrey M. Wooldridge
摘要
Consider a researcher estimating the parameters of a regression function based on data for all 50 states in the United States or on data for all visits to a website. What is the interpretation of the estimated parameters and the standard errors? In practice, researchers typically assume that the sample is randomly drawn from a large population of interest and report standard errors that are designed to capture sampling variation. This is common even in applications where it is difficult to articulate what that population of interest is, and how it differs from the sample. In this article, we explore an alternative approach to inference, which is partly design‐based. In a design‐based setting, the values of some of the regressors can be manipulated, perhaps through a policy intervention. Design‐based uncertainty emanates from lack of knowledge about the values that the regression outcome would have taken under alternative interventions. We derive standard errors that account for design‐based uncertainty instead of, or in addition to, sampling‐based uncertainty. We show that our standard errors in general are smaller than the usual infinite‐population sampling‐based standard errors and provide conditions under which they coincide.
JMLR High-Dimensional · 发表于 2020
Chinot Geoffrey, Lecué Guillaume, Lerasle Matthieu
摘要
We establish risk bounds for Regularized Empirical Risk Minimizers (RERM) when the loss is Lipschitz and convex and the regularization function is a norm. In a first part, we obtain these results in the i.i.d. setup under subgaussian assumptions on the design. In a second part, a more general framework where the design might have heavier tails and data may be corrupted by outliers both in the design and the response variables is considered. In this situation, RERM performs poorly in general. We analyse an alternative procedure based on median-of-means principles and called “minmax MOM”. We show optimal subgaussian deviation rates for these estimators in the relaxed setting. The main results are meta-theorems allowing a wide-range of applications to various problems in learning theory. To show a non-exhaustive sample of these potential applications, it is applied to classification problems with logistic loss functions regularized by LASSO and SLOPE, to regression problems with Huber loss regularized by Group LASSO and Total Variation. Another advantage of the minmax MOM formulation is that it suggests a systematic way to slightly modify descent based algorithms used in high-dimensional statistics to make them robust to outliers. We illustrate this principle in a Simulations section where a “ minmax MOM” version of classical proximal descent algorithms are turned into robust to outliers algorithms.
JMLR Bayesian Methods Optimization & Learning Theory Machine Learning · 发表于 2020
Supratik Paul, Konstantinos Chatzilygeroudis, Kamil Ciosek, Jean-Baptiste Mouret, Michael A. Osborne, Shimon Whiteson
摘要
Bayesian optimisation has been successfully applied to a variety of reinforcement learning problems. However, the traditional approach for learning optimal policies in simulators does not utilise the opportunity to improve learning by adjusting certain environment variables: state features that are unobservable and randomly determined by the environment in a physical setting but are controllable in a simulator. This article considers the problem of finding a robust policy while taking into account the impact of environment variables. We present Alternating Optimisation and Quadrature (ALOQ), which uses Bayesian optimisation and Bayesian quadrature to address such settings. We also present Transferable ALOQ (TALOQ), for settings where simulator inaccuracies lead to difficulty in transferring the learnt policy to the physical system. We show that our algorithms are robust to the presence of significant rare events, which may not be observable under random sampling but play a substantial role in determining the optimal policy. Experimental results across different domains show that our algorithms learn robust policies efficiently.
JMLR Optimization & Learning Theory Networks & Graphs Meta-Analysis & Data Integration · 发表于 2020
Artin Spiridonoff, Alex Olshevsky, Ioannis Ch. Paschalidis
摘要
We consider the standard model of distributed optimization of a sum of functions $F(\mathbf z) = \sum_{i=1}^n f_i(\mathbf z)$, where node $i$ in a network holds the function $f_i(\mathbf z)$. We allow for a harsh network model characterized by asynchronous updates, message delays, unpredictable message losses, and directed communication among nodes. In this setting, we analyze a modification of the Gradient-Push method for distributed optimization, assuming that (i) node $i$ is capable of generating gradients of its function $f_i(\mathbf z)$ corrupted by zero-mean bounded-support additive noise at each step, (ii) $F(\mathbf z)$ is strongly convex, and (iii) each $f_i(\mathbf z)$ has Lipschitz gradients. We show that our proposed method asymptotically performs as well as the best bounds on centralized gradient descent that takes steps in the direction of the sum of the noisy gradients of all the functions $f_1(\mathbf z), \ldots, f_n(\mathbf z)$ at each step.
JMLR Other · 发表于 2020
Lukas Gonon, Lyudmila Grigoryeva, Juan-Pablo Ortega
摘要
We analyze the practices of reservoir computing in the framework of statistical learning theory. In particular, we derive finite sample upper bounds for the generalization error committed by specific families of reservoir computing systems when processing discrete-time inputs under various hypotheses on their dependence structure. Non-asymptotic bounds are explicitly written down in terms of the multivariate Rademacher complexities of the reservoir systems and the weak dependence structure of the signals that are being handled. This allows, in particular, to determine the minimal number of observations needed in order to guarantee a prescribed estimation accuracy with high probability for a given reservoir family. At the same time, the asymptotic behavior of the devised bounds guarantees the consistency of the empirical risk minimization procedure for various hypothesis classes of reservoir functionals.
Econometrica Other · 发表于 2020
Yuichiro Kamada, Michihiro Kandori
摘要
This paper proposes a class of games called revision games. In a revision game, players start with initially prepared actions, followed by a sequence of random revision opportunities until a predetermined deadline. In the course of revisions, players monitor each other's behavior. It is shown that players can cooperate and that their behavior under the optimal equilibrium is described by a simple differential equation. We present the necessary and sufficient conditions for cooperation to be sustained in revision games. We also present applications to the preopening activities in the stock exchange and to an electoral campaign.
Biometrics Genetics & Omics · 发表于 2020
Yukun Liu, Pengfei Li, Lei Song, Kai Yu, Jing Qin
摘要
Since the seminal work of Prentice and Pyke, the prospective logistic likelihood has become the standard method of analysis for retrospectively collected case‐control data, in particular for testing the association between a single genetic marker and a disease outcome in genetic case‐control studies. In the study of multiple genetic markers with relatively small effects, especially those with rare variants, various aggregated approaches based on the same prospective likelihood have been developed to integrate subtle association evidence among all the markers considered. Many of the commonly used tests are derived from the prospective likelihood under a common‐random‐effect assumption, which assumes a common random effect for all subjects. We develop the locally most powerful aggregation test based on the retrospective likelihood under an independent‐random‐effect assumption, which allows the genetic effect to vary among subjects. In contrast to the fact that disease prevalence information cannot be used to improve efficiency for the estimation of odds ratio parameters in logistic regression models, we show that it can be utilized to enhance the testing power in genetic association studies. Extensive simulations demonstrate the advantages of the proposed method over the existing ones. A real genome‐wide association study is analyzed for illustration.
Econometrica Econometrics & Economics · 发表于 2020
Eduardo Faingold
摘要
Equilibrium payoff bounds from reputation effects are derived for repeated games with imperfect public monitoring in which a long‐run player interacts frequently with a population of short‐run players and the monitoring technology scales with the length of the period of interaction. The bounds depend on the monitoring technology through the flow of information , a measure of signal informativeness per unit of time based on relative entropy. Examples are shown where, under complete information, the set of equilibrium payoffs of the long‐run player converges, as the period length tends to zero, to the set of static equilibrium payoffs, whereas when the game is perturbed by a small ex ante probability on commitment types, reputation effects remain powerful in the high‐frequency limit.
Econometrica Econometrics & Economics · 发表于 2020
Harry Pei
摘要
A patient player privately observes a persistent state and interacts with an infinite sequence of myopic uninformed players. The patient player is either a strategic type who maximizes his payoff or one of several commitment types who mechanically play the same action in every period. I focus on situations in which the uninformed player's best reply to a commitment action depends on the state and where the total probability of commitment types is sufficiently small. I show that the patient player's equilibrium payoff is bounded below his commitment payoff in some equilibria under some of his payoff functions. This is because he faces a trade‐off between building his reputation for commitment and signaling favorable information about the state. When players' stage‐game payoff functions are monotone‐supermodular, the patient player receives high payoffs in all states and in all equilibria. Under an additional condition on the state distribution, my reputation model yields a unique prediction on the patient player's equilibrium payoff and on‐path behavior.
JMLR Networks & Graphs Survey & Sampling · 发表于 2020
Seyed Mehran Kazemi, Rishab Goel, Kshitij Jain, Ivan Kobyzev, Akshay Sethi, Peter Forsyth, Pascal Poupart
摘要
Graphs arise naturally in many real-world applications including social networks, recommender systems, ontologies, biology, and computational finance. Traditionally, machine learning models for graphs have been mostly designed for static graphs. However, many applications involve evolving graphs. This introduces important challenges for learning and inference since nodes, attributes, and edges change over time. In this survey, we review the recent advances in representation learning for dynamic graphs, including dynamic knowledge graphs. We describe existing models from an encoder-decoder perspective, categorize these encoders and decoders based on the techniques they employ, and analyze the approaches in each category. We also review several prominent applications and widely used datasets and highlight directions for future research.
JMLR Machine Learning · 发表于 2020
Haoran Wang, Thaleia Zariphopoulou, Xun Yu Zhou
摘要
We consider reinforcement learning (RL) in continuous time with continuous feature and action spaces. We motivate and devise an exploratory formulation for the feature dynamics that captures learning under exploration, with the resulting optimization problem being a revitalization of the classical relaxed stochastic control. We then study the problem of achieving the best trade-off between exploration and exploitation by considering an entropy-regularized reward function. We carry out a complete analysis of the problem in the linear--quadratic (LQ) setting and deduce that the optimal feedback control distribution for balancing exploitation and exploration is Gaussian. This in turn interprets the widely adopted Gaussian exploration in RL, beyond its simplicity for sampling. Moreover, the exploitation and exploration are captured respectively by the mean and variance of the Gaussian distribution. We characterize the cost of exploration, which, for the LQ case, is shown to be proportional to the entropy regularization weight and inversely proportional to the discount rate. Finally, as the weight of exploration decays to zero, we prove the convergence of the solution of the entropy-regularized LQ problem to the one of the classical LQ problem.
JMLR Other · 发表于 2020
Francois Kamper, Sarel J. Steel, Johan A. du Preez
摘要
Gaussian belief propagation (GaBP) is a message-passing algorithm that can be used to perform approximate inference on a pairwise Markov graph (MG) constructed from a multivariate Gaussian distribution in canonical parameterization. The output of GaBP is a set of approximate univariate marginals for each variable in the pairwise MG. An extension of GaBP (labeled GaBP-m), allowing for the approximation of higher-dimensional marginal distributions, was explored by Kamper et al. (2019). The idea is to create an MG in which each node is allowed to receive more than one variable. As in the univariate case, the multivariate extension does not necessarily converge in loopy graphs and, even if convergence occurs, is not guaranteed to provide exact inference. To address the problem of convergence, we consider a multivariate extension of the principle of node regularization proposed by Kamper et al. (2018). We label this algorithm slow GaBP-m (sGaBP-m), where the term 'slow' relates to the damping effect of the regularization on the message passing. We prove that, given sufficient regularization, this algorithm will converge and provide the exact marginal means at convergence, regardless of the way variables are assigned to nodes. The selection of the degree of regularization is addressed through the use of a heuristic, which is based on a tree representation of sGaBP-m. As a further contribution, we extend other GaBP variants in the literature to allow for higher-dimensional marginalization. We show that our algorithm compares favorably with these variants, both in terms of convergence speed and inference quality.
JMLR High-Dimensional Spatial & Time Series · 发表于 2020
Jiahe Lin, George Michailidis
摘要
A factor-augmented vector autoregressive (FAVAR) model is defined by a VAR equation that captures lead-lag correlations amongst a set of observed variables $X$ and latent factors $F$, and a calibration equation that relates another set of observed variables $Y$ with $F$ and $X$. The latter equation is used to estimate the factors that are subsequently used in estimating the parameters of the VAR system. The FAVAR model has become popular in applied economic research, since it can summarize a large number of variables of interest as a few factors through the calibration equation and subsequently examine their influence on core variables of primary interest through the VAR equation. However, there is increasing need for examining lead-lag relationships between a large number of time series, while incorporating information from another high-dimensional set of variables. Hence, in this paper we investigate the FAVAR model under high-dimensional scaling. We introduce an appropriate identification constraint for the model parameters, which when incorporated into the formulated optimization problem yields estimates with good statistical properties. Further, we address a number of technical challenges introduced by the fact that estimates of the VAR system model parameters are based on estimated rather than directly observed quantities. The performance of the proposed estimators is evaluated on synthetic data. Further, the model is applied to commodity prices and reveals interesting and interpretable relationships between the prices and the factors extracted from a set of global macroeconomic indicators.
JMLR Other · 发表于 2020
Yichong Xu, Sivaraman Balakrishnan, Aarti Singh, Artur Dubrawski
摘要
In supervised learning, we typically leverage a fully labeled dataset to design methods for function estimation or prediction. In many practical situations, we are able to obtain alternative feedback, possibly at a low cost. A broad goal is to understand the usefulness of, and to design algorithms to exploit, this alternative feedback. In this paper, we consider a semi-supervised regression setting, where we obtain additional ordinal (or comparison) information for the unlabeled samples. We consider ordinal feedback of varying qualities where we have either a perfect ordering of the samples, a noisy ordering of the samples or noisy pairwise comparisons between the samples. We provide a precise quantification of the usefulness of these types of ordinal feedback in both nonparametric and linear regression, showing that in many cases it is possible to accurately estimate an underlying function with a very small labeled set, effectively escaping the curse of dimensionality. We also present lower bounds, that establish fundamental limits for the task and show that our algorithms are optimal in a variety of settings. Finally, we present extensive experiments on new datasets that demonstrate the efficacy and practicality of our algorithms and investigate their robustness to various sources of noise and model misspecification.
JMLR Latent Variables & Mixtures · 发表于 2020
Tao Jiang, Stephen Vavasis, Chen Wen Zhai
摘要
Sum-of-norms clustering is a method for assigning $n$ points in $\mathbf{R}^d$ to $K$ clusters, $1\le K\le n$, using convex optimization. Recently, Panahi (2017) proved that sum-of-norms clustering is guaranteed to recover a mixture of Gaussians under the restriction that the number of samples is not too large. The purpose of this note is to lift this restriction, that is, show that sum-of-norms clustering can recover a mixture of Gaussians even as the number of samples tends to infinity. Our proof relies on an interesting characterization of clusters computed by sum-of-norms clustering that was developed inside a proof of the agglomeration conjecture by Chiquet et al. (2017). Because we believe this theorem has independent interest, we restate and reprove the Chiquet et al. (2017) result herein.
Econometrica Other · 发表于 2020
Tim Bollerslev, Jia Li, Andrew J. Patton, Rogier Quaedvlieg
摘要
We propose a decomposition of the realized covariance matrix into components based on the signs of the underlying high‐frequency returns, and we derive the asymptotic properties of the resulting realized semicovariance measures as the sampling interval goes to zero. The first‐order asymptotic results highlight how the same‐sign and mixed‐sign components load differently on economic information related to stochastic correlation and jumps. The second‐order asymptotic results reveal the structure underlying the same‐sign semicovariances, as manifested in the form of co‐drifting and dynamic “leverage” effects. In line with this anatomy, we use data on a large cross‐section of individual stocks to empirically document distinct dynamic dependencies in the different realized semicovariance components. We show that the accuracy of portfolio return variance forecasts may be significantly improved by exploiting the information in realized semicovariances.
JMLR Machine Learning Bayesian Methods · 发表于 2020
William Hoiles, Vikram Krishnamurthy, Kunal Pattanayak
摘要
We consider a novel application of inverse reinforcement learning with behavioral economics constraints to model, learn and predict the commenting behavior of YouTube viewers. Each group of users is modeled as a rationally inattentive Bayesian agent which solves a contextual bandit problem. Our methodology integrates three key components. First, to identify distinct commenting patterns, we use deep embedded clustering to estimate framing information (essential extrinsic features) that clusters users into distinct groups. Second, we present an inverse reinforcement learning algorithm that uses Bayesian revealed preferences to test for rationality: does there exist a utility function that rationalizes the given data, and if yes, can it be used to predict commenting behavior? Finally, we impose behavioral economics constraints stemming from rational inattention to characterize the attention span of groups of users. The test imposes a Renyi mutual information cost constraint which impacts how the agent can select attention strategies to maximize their expected utility. After a careful analysis of a massive YouTube dataset, our surprising result is that in most YouTube user groups, the commenting behavior is consistent with optimizing a Bayesian utility with rationally inattentive constraints. The paper also highlights how the rational inattention model can accurately predict commenting behavior. The massive YouTube dataset and analysis used in this paper are available on GitHub and completely reproducible.
Econometrica Econometrics & Economics · 发表于 2020
David Domeij, Tore Ellingsen
摘要
Giglio, Maggiori, and Stroebel (2016) show that there is no significant price difference between freeholds and ultra‐long leaseholds in the UK housing market. They claim that this finding precludes the presence of large rational bubbles, as these can only attach to the price of freeholds. But the conclusion presumes that leaseholders cannot acquire bubbles through enfranchisement at favorable prices. We find that the presumption is violated. Enfranchisement rights are comprehensive and cheap to exercise. We also dispute the counter‐argument that cheap enfranchisement proves that market participants, if they have rational expectations, must have explicitly concluded that freehold prices are bubbleless.
JMLR High-Dimensional Nonparametric & Semiparametric · 发表于 2020
Wojciech Rejchel, Małgorzata Bogdan
摘要
We consider the problem of identifying significant predictors in large data bases, where the response variable depends on the linear combination of explanatory variables through an unknown monotonic link function, corrupted with the noise from the unknown distribution. We utilize the natural, robust and efficient approach, which relies on replacing values of the response variables by their ranks and then identifying significant predictors by using well known Lasso. We provide new consistency results for the proposed procedure (called ,,RankLasso”) and extend the scope of its applications by proposing its thresholded and adaptive versions. Our theoretical results show that these modifications can identify the set of relevant predictors under a wide range of data generating scenarios. Theoretical results are supported by the simulation study and the real data analysis, which show that our methods can properly identify relevant predictors, even when the error terms come from the Cauchy distribution and the link function is nonlinear. They also demonstrate the superiority of the modified versions of RankLasso over its regular version in the case when predictors are substantially correlated. The numerical study shows also that RankLasso performs substantially better in model selection than LADLasso, which is a well established methodology for robust model selection.
JMLR Machine Learning · 发表于 2020
Lucas Mentch, Siyu Zhou
摘要
Random forests remain among the most popular off-the-shelf supervised machine learning tools with a well-established track record of predictive accuracy in both regression and classification settings. Despite their empirical success as well as a bevy of recent work investigating their statistical properties, a full and satisfying explanation for their success has yet to be put forth. Here we aim to take a step forward in this direction by demonstrating that the additional randomness injected into individual trees serves as a form of implicit regularization, making random forests an ideal model in low signal-to-noise ratio (SNR) settings. Specifically, from a model-complexity perspective, we show that the mtry parameter in random forests serves much the same purpose as the shrinkage penalty in explicitly regularized regression procedures like lasso and ridge regression. To highlight this point, we design a randomized linear-model-based forward selection procedure intended as an analogue to tree-based random forests and demonstrate its surprisingly strong empirical performance. Numerous demonstrations on both real and synthetic data are provided.
Econometrica Other · 发表于 2020
Shaowei Ke, Qi Zhang
摘要
We propose a model of preferences in which the effect of randomization on ambiguity depends on how the unknown probability law is determined. We adopt the framework of Anscombe and Aumann (1963) and relax the axioms. In the resulting representation of the individual's preference, the individual has a collection of sets of priors M . She believes that before she moves, nature has chosen an unknown scenario (a set of priors) from M , and from that scenario, nature will choose a prior after she moves. The representation illustrates how randomization may partially eliminate the effect of ambiguity.
JMLR High-Dimensional · 发表于 2020
Avrim Blum, Travis Dick, Naren Manoj, Hongyang Zhang
摘要
We show a hardness result for random smoothing to achieve certified adversarial robustness against attacks in the $\ell_p$ ball of radius $\epsilon$ when $p>2$. Although random smoothing has been well understood for the $\ell_2$ case using the Gaussian distribution, much remains unknown concerning the existence of a noise distribution that works for the case of $p>2$. This has been posed as an open problem by Cohen et al. (2019) and includes many significant paradigms such as the $\ell_\infty$ threat model. In this work, we show that any noise distribution $\mathcal{D}$ over $\mathbb{R}^d$ that provides $\ell_p$ robustness for all base classifiers with $p>2$ must satisfy $\mathbb{E} \eta_i^2=\Omega(d^{1-2/p}\epsilon^2(1-\delta)/\delta^2)$ for 99% of the features (pixels) of vector $\eta\sim\mathcal{D}$, where $\epsilon$ is the robust radius and $\delta$ is the score gap between the highest-scored class and the runner-up. Therefore, for high-dimensional images with pixel values bounded in $[0,255]$, the required noise will eventually dominate the useful information in the images, leading to trivial smoothed classifiers.
JMLR Networks & Graphs Bayesian Methods · 发表于 2020
Nilabja Guha, Veera Baladandayuthapani, Bani K. Mallick
摘要
Graphical models are ubiquitous tools to describe the interdependence between variables measured simultaneously such as large-scale gene or protein expression data. Gaussian graphical models (GGMs) are well-established tools for probabilistic exploration of dependence structures using precision matrices and they are generated under a multivariate normal joint distribution. However, they suffer from several shortcomings since they are based on Gaussian distribution assumptions. In this article, we propose a Bayesian quantile based approach for sparse estimation of graphs. We demonstrate that the resulting graph estimation is robust to outliers and applicable under general distributional assumptions. Furthermore, we develop efficient variational Bayes approximations to scale the methods for large data sets. Our methods are applied to a novel cancer proteomics data dataset where-in multiple proteomic antibodies are simultaneously assessed on tumor samples using reverse-phase protein arrays (RPPA) technology.
JMLR Other · 发表于 2020
Pan Li, Niao He, Olgica Milenkovic
摘要
We introduce a new convex optimization problem, termed quadratic decomposable submodular function minimization (QDSFM), which allows to model a number of learning tasks on graphs and hypergraphs. The problem exhibits close ties to decomposable submodular function minimization (DSFM) yet is much more challenging to solve. We approach the problem via a new dual strategy and formulate an objective that can be optimized through a number of double-loop algorithms. The outer-loop uses either random coordinate descent (RCD) or alternative projection (AP) methods, for both of which we prove linear convergence rates. The inner-loop computes projections onto cones generated by base polytopes of the submodular functions via the modified min-norm-point or Frank-Wolfe algorithms. We also describe two new applications of QDSFM: hypergraph-adapted PageRank and semi-supervised learning. The proposed hypergraph-based PageRank algorithm can be used for local hypergraph partitioning and comes with provable performance guarantees. For hypergraph-adapted semi-supervised learning, we provide numerical experiments demonstrating the efficiency of our QDSFM solvers and their significant improvements on prediction accuracy when compared to state-of-the-art methods.
JMLR Optimization & Learning Theory · 发表于 2020
Nhan H. Pham, Lam M. Nguyen, Dzung T. Phan, Quoc Tran-Dinh
摘要
We propose a new stochastic first-order algorithmic framework to solve stochastic composite nonconvex optimization problems that covers both finite-sum and expectation settings. Our algorithms rely on the SARAH estimator and consist of two steps: a proximal gradient and an averaging step making them different from existing nonconvex proximal-type algorithms. The algorithms only require an average smoothness assumption of the nonconvex objective term and additional bounded variance assumption if applied to expectation problems. They work with both constant and dynamic step-sizes, while allowing single sample and mini-batches. In all these cases, we prove that our algorithms can achieve the best-known complexity bounds in terms of stochastic first-order oracle. One key step of our methods is the new constant and dynamic step-sizes resulting in the desired complexity bounds while improving practical performance. Our constant step-size is much larger than existing methods including proximal SVRG scheme in the single sample case. We also specify our framework to the non-composite case that covers existing state-of-the-arts in terms of oracle complexity bounds. Our update also allows one to trade-off between step-sizes and mini-batch sizes to improve performance. We test the proposed algorithms on two composite nonconvex problems and neural networks using several well-known data sets.
JMLR Other · 发表于 2020
Mihai Cucuringu, Hemant Tyagi
摘要
Consider an unknown smooth function $f: [0,1]^d \rightarrow \mathbb{R}$, and assume we are given $n$ noisy mod 1 samples of $f$, i.e., $y_i = (f(x_i) + \eta_i) \bmod 1$, for $x_i \in [0,1]^d$, where $\eta_i$ denotes the noise. Given the samples $(x_i,y_i)_{i=1}^{n}$, our goal is to recover smooth, robust estimates of the clean samples $f(x_i) \bmod 1$. We formulate a natural approach for solving this problem, which works with angular embeddings of the noisy mod 1 samples over the unit circle, inspired by the angular synchronization framework. This amounts to solving a smoothness regularized least-squares problem -- a quadratically constrained quadratic program (QCQP) -- where the variables are constrained to lie on the unit circle. Our proposed approach is based on solving its relaxation, which is a trust-region sub-problem and hence solvable efficiently. We provide theoretical guarantees demonstrating its robustness to noise for adversarial, as well as random Gaussian and Bernoulli noise models. To the best of our knowledge, these are the first such theoretical results for this problem. We demonstrate the robustness and efficiency of our proposed approach via extensive numerical simulations on synthetic data, along with a simple least-squares based solution for the unwrapping stage, that recovers the original samples of $f$ (up to a global shift). It is shown to perform well at high levels of noise, when taking as input the denoised modulo $1$ samples. Finally, we also consider two other approaches for denoising the modulo 1 samples that leverage tools from Riemannian optimization on manifolds, including a Burer-Monteiro approach for a semidefinite programming relaxation of our formulation. For the two-dimensional version of the problem, which has applications in synthetic aperture radar interferometry (InSAR), we are able to solve instances of real-world data with a million sample points in under 10 seconds, on a personal laptop.
JMLR Latent Variables & Mixtures · 发表于 2020
Eric C. Chi, Brian J. Gaines, Will Wei Sun, Hua Zhou, Jian Yang
摘要
Cluster analysis is a fundamental tool for pattern discovery of complex heterogeneous data. Prevalent clustering methods mainly focus on vector or matrix-variate data and are not applicable to general-order tensors, which arise frequently in modern scientific and business applications. Moreover, there is a gap between statistical guarantees and computational efficiency for existing tensor clustering solutions due to the nature of their non-convex formulations. In this work, we bridge this gap by developing a provable convex formulation of tensor co-clustering. Our convex co-clustering (CoCo) estimator enjoys stability guarantees and its computational and storage costs are polynomial in the size of the data. We further establish a non-asymptotic error bound for the CoCo estimator, which reveals a surprising “blessing of dimensionality” phenomenon that does not exist in vector or matrix-variate cluster analysis. Our theoretical findings are supported by extensive simulated studies. Finally, we apply the CoCo estimator to the cluster analysis of advertisement click tensor data from a major online company. Our clustering results provide meaningful business insights to improve advertising effectiveness.
JMLR Other · 发表于 2020
Sercan O. Arik, Tomas Pfister
摘要
We propose a novel inherently interpretable machine learning method that bases decisions on few relevant examples that we call prototypes. Our method, ProtoAttend, can be integrated into a wide range of neural network architectures including pre-trained models. It utilizes an attention mechanism that relates the encoded representations to samples in order to determine prototypes. Protoattend yields superior results in three high impact problems without sacrificing accuracy of the original model: (1)it enables high-quality interpretability that outputs samples most relevant to the decision-making (i.e. a sample-based interpretability method); (2) it achieves state of the art confidence estimation by quantifying the mismatch across prototype labels; and (3) it obtains state of the art in distribution mismatch detection. All these can be achieved with minimal additional test time and a practically viable training time computational cost.
JMLR Networks & Graphs Machine Learning · 发表于 2020
Benjamin Bloem-Reddy, Yee Whye Teh
摘要
Treating neural network inputs and outputs as random variables, we characterize the structure of neural networks that can be used to model data that are invariant or equivariant under the action of a compact group. Much recent research has been devoted to encoding invariance under symmetry transformations into neural network architectures, in an effort to improve the performance of deep neural networks in data-scarce, non-i.i.d., or unsupervised settings. By considering group invariance from the perspective of probabilistic symmetry, we establish a link between functional and probabilistic symmetry, and obtain generative functional representations of probability distributions that are invariant or equivariant under the action of a compact group. Our representations completely characterize the structure of neural networks that can be used to model such distributions and yield a general program for constructing invariant stochastic or deterministic neural networks. We demonstrate that examples from the recent literature are special cases, and develop the details of the general program for exchangeable sequences and arrays.