2022 年的期刊文章 · 第 32 页

共 1481 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2022 到 2022。

按期刊、按主题看 →

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 页

Econometrica Econometrics & Economics · 发表于 2022

Monetary Policy, Redistribution, and Risk Premia

Rohan Kekre, Moritz Lenel

摘要

We study the transmission of monetary policy through risk premia in a heterogeneous agent New Keynesian environment. Heterogeneity in households' marginal propensity to take risk (MPR) summarizes differences in portfolio choice on the margin. An unexpected reduction in the nominal interest rate redistributes to households with high MPRs, lowering risk premia and amplifying the stimulus to the real economy. Quantitatively, this mechanism rationalizes the role of news about future excess returns in driving the stock market response to monetary policy shocks and amplifies their real effects by 1.3–1.4 times.

Econometrica Other · 发表于 2022

Model and Predictive Uncertainty: A Foundation for Smooth Ambiguity Preferences

Tommaso Denti, Luciano Pomatto

摘要

Smooth ambiguity preferences (Klibanoff, Marinacci, and Mukerji (2005)) describe a decision maker who evaluates each act f according to the twofold expectation V ( f ) = ∫ P ϕ ( ∫ Ω u ( f ) d p ) d μ ( p ) defined by a utility function u, an ambiguity index ϕ, and a belief μ over a set P of probabilities. We provide an axiomatic foundation for the representation, taking as a primitive a preference over Anscombe–Aumann acts. We study a special case where P is a subjective statistical model that is point identified, that is, the decision maker believes that the true law p ∈ P can be recovered empirically. Our main axiom is a joint weakening of Savage's sure‐thing principle and Anscombe–Aumann's mixture independence. In addition, we show that the parameters of the representation can be uniquely recovered from preferences, thereby making operational the separation between ambiguity attitude and perception, a hallmark feature of the smooth ambiguity representation.

JMLR Other · 发表于 2022

Model Averaging Is Asymptotically Better Than Model Selection For Prediction

Tri M. Le, Bertrand S. Clarke

摘要

We compare the performance of six model average predictors---Mallows' model averaging, stacking, Bayes model averaging, bagging, random forests, and boosting---to the components used to form them.In all six cases we identify conditions under which the model average predictor is consistent for its intended limit and performs as well or better than any of its components asymptotically. This is well known empirically, especially for complex problems, although theoretical results do not seem to have been formally established. We have focused our attention on the regression context since that is wheremodel averaging techniques differ most often from current practice.

JMLR Networks & Graphs Latent Variables & Mixtures Bayesian Methods · 发表于 2022

Mitigating the Effects of Non-Identifiability on Inference for Bayesian Neural Networks with Latent Variables

Yaniv Yacoby, Weiwei Pan, Finale Doshi-Velez

摘要

Bayesian Neural Networks with Latent Variables (BNN+LVs) capture predictive uncertainty by explicitly modeling model uncertainty (via priors on network weights) and environmental stochasticity (via a latent input noise variable). In this work, we first show that BNN+LV suffers from a serious form of non-identifiability: explanatory power can be transferred between the model parameters and latent variables while fitting the data equally well. We demonstrate that as a result, in the limit of infinite data, the posterior mode over the network weights and latent variables is asymptotically biased away from the ground-truth. Due to this asymptotic bias, traditional inference methods may in practice yield parameters that generalize poorly and misestimate uncertainty. Next, we develop a novel inference procedure that explicitly mitigates the effects of likelihood non-identifiability during training and yields high-quality predictions as well as uncertainty estimates. We demonstrate that our inference method improves upon benchmark methods across a range of synthetic and real data-sets.

Econometrica Longitudinal Data Econometrics & Economics · 发表于 2022

Misallocation, Selection, and Productivity: A Quantitative Analysis With Panel Data From China

Tasso Adamopoulos, Loren Brandt, Jessica Leight, Diego Restuccia

摘要

We use household‐level panel data from China and a quantitative framework to document the extent and consequences of factor misallocation in agriculture. We find that there are substantial within‐village frictions in both the land and capital markets linked to land institutions in rural China that disproportionately constrain the more productive farmers. These frictions reduce aggregate agricultural productivity by affecting two key margins: (1) the allocation of resources across farmers (misallocation) and (2) the allocation of workers across sectors, in particular the type of farmers who operate in agriculture (selection). Selection substantially amplifies the productivity effect of distortionary policies by affecting occupational choices that worsen average ability in agriculture.

JMLR Other · 发表于 2022

Minimax optimal approaches to the label shift problem in non-parametric settings

Subha Maity, Yuekai Sun, Moulinath Banerjee

摘要

We study the minimax rates of the label shift problem in non-parametric classification. In addition to the unsupervised setting in which the learner only has access to unlabeled examples from the target domain, we also consider the setting in which a small number of labeled examples from the target domain is available to the learner. Our study reveals a difference in the difficulty of the label shift problem in the two settings, and we attribute this difference to the availability of data from the target domain to estimate the class conditional distributions in the latter setting. We also show that a class proportion estimation approach is minimax rate-optimal in the unsupervised setting.

JMLR Other · 发表于 2022

Minimax Mixing Time of the Metropolis-Adjusted Langevin Algorithm for Log-Concave Sampling

Keru Wu, Scott Schmidler, Yuansi Chen

摘要

We study the mixing time of the Metropolis-adjusted Langevin algorithm (MALA) for sampling from a log-smooth and strongly log-concave distribution. We establish its optimal minimax mixing time under a warm start. Our main contribution is two-fold. First, for a $d$-dimensional log-concave density with condition number $\kappa$, we show that MALA with a warm start mixes in $\tilde O(\kappa \sqrt{d})$ iterations up to logarithmic factors. This improves upon the previous work on the dependency of either the condition number $\kappa$ or the dimension $d$. Our proof relies on comparing the leapfrog integrator with the continuous Hamiltonian dynamics, where we establish a new concentration bound for the acceptance rate. Second, we prove a spectral gap based mixing time lower bound for reversible MCMC algorithms on general state spaces. We apply this lower bound result to construct a hard distribution for which MALA requires at least $\tilde\Omega(\kappa \sqrt{d})$ steps to mix. The lower bound for MALA matches our upper bound in terms of condition number and dimension. Finally, numerical experiments are included to validate our theoretical results.

JMLR Diagnostics & Prediction Nonparametric & Semiparametric Networks & Graphs · 发表于 2022

Metrics of Calibration for Probabilistic Predictions

Imanol Arrieta-Ibarra, Paman Gujral, Jonathan Tannen, Mark Tygert, Cherie Xu

摘要

Many predictions are probabilistic in nature; for example, a prediction could be for precipitation tomorrow, but with only a 30 percent chance. Given such probabilistic predictions together with the actual outcomes, “reliability diagrams” (also known as “calibration plots”) help detect and diagnose statistically significant discrepancies—so-called “miscalibration”—between the predictions and the outcomes. The canonical reliability diagrams are based on histogramming the observed and expected values of the predictions; replacing the hard histogram binning with soft kernel density estimation using smooth convolutional kernels is another common practice. But, which widths of bins or kernels are best? Plots of the cumulative differences between the observed and expected values largely avoid this question, by displaying miscalibration directly as the slopes of secant lines for the graphs. Slope is easy to perceive with quantitative precision, even when the constant offsets of the secant lines are irrelevant; there is no need to bin or perform kernel density estimation. The existing standard metrics of miscalibration each summarize a reliability diagram into a single scalar statistic. The cumulative plots naturally lead to scalar metrics for the deviation of the graph of cumulative differences away from zero; good calibration corresponds to a horizontal, flat graph which deviates little from zero. The cumulative approach is currently unconventional, yet offers many favorable statistical properties, guaranteed via mathematical theory backed by rigorous proofs and illustrative numerical examples. In particular, metrics based on binning or kernel density estimation unavoidably must trade-off statistical confidence for the ability to resolve variations as a function of the predicted probability or vice versa. Widening the bins or kernels averages away random noise while giving up some resolving power. Narrowing the bins or kernels enhances resolving power while not averaging away as much noise. The cumulative methods do not impose such an explicit trade-off. Considering these results, practitioners probably should adopt the cumulative approach as a standard for best practices.

JMLR High-Dimensional Meta-Analysis & Data Integration · 发表于 2022

Meta-analysis of heterogeneous data: integrative sparse regression in high-dimensions

Subha Maity, Yuekai Sun, Moulinath Banerjee

摘要

We consider the task of meta-analysis in high-dimensional settings in which the data sources are similar but non-identical. To borrow strength across such heterogeneous datasets, we introduce a global parameter that emphasizes interpretability and statistical efficiency in the presence of heterogeneity. We also propose a one-shot estimator of the global parameter that preserves the anonymity of the data sources and converges at a rate that depends on the size of the combined dataset. For high-dimensional linear model settings, we demonstrate the superiority of our identification restrictions in adapting to a previously seen data distribution as well as predicting for a new/unseen data distribution. Finally, we demonstrate the benefits of our approach on a large-scale drug treatment dataset involving several different cancer cell-lines.

Econometrica Other · 发表于 2022

Media Competition and Social Disagreement

Jacopo Perego, Sevgi Yuksel

摘要

We study the competitive provision and endogenous acquisition of political information. Our main result identifies a natural equilibrium channel through which a more competitive market decreases the efficiency of policy outcomes. A critical insight we put forward is that competition among information providers leads to informational specialization: firms provide relatively less information on issues that are of common interest and relatively more information on issues on which agents' preferences are heterogeneous. This enables agents to acquire information about different aspects of the policy, specifically, those that are particularly important to them. This leads to an increase in social disagreement, which has negative welfare implications. We establish that, in large enough societies, competition makes every agent worse off by decreasing the utility that she derives from the policy outcome. Furthermore, we show that this decline cannot be compensated by the decrease in prices resulting from competition.

Econometrica Econometrics & Economics · 发表于 2022

Mechanism Design With Limited Commitment

Laura Doval, Vasiliki Skreta

摘要

We develop a tool akin to the revelation principle for dynamic mechanism‐selection games in which the designer can only commit to short‐term mechanisms. We identify a canonical class of mechanisms rich enough to replicate the outcomes of any equilibrium in a mechanism‐selection game between an uninformed designer and a privately informed agent. A cornerstone of our methodology is the idea that a mechanism should encode not only the rules that determine the allocation, but also the information the designer obtains from the interaction with the agent. Therefore, how much the designer learns, which is the key tension in design with limited commitment, becomes an explicit part of the design. Our result simplifies the search for the designer‐optimal outcome by reducing the agent's behavior to a series of participation, truth telling, and Bayes' plausibility constraints the mechanisms must satisfy.

JMLR Networks & Graphs · 发表于 2022

Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks

Alexander Shevchenko, Vyacheslav Kungurtsev, Marco Mondelli

摘要

Understanding the properties of neural networks trained via stochastic gradient descent (SGD) is at the heart of the theory of deep learning. In this work, we take a mean-field view, and consider a two-layer ReLU network trained via noisy-SGD for a univariate regularized regression problem. Our main result is that SGD with vanishingly small noise injected in the gradients is biased towards a simple solution: at convergence, the ReLU network implements a piecewise linear map of the inputs, and the number of “knot” points -- i.e., points where the tangent of the ReLU network estimator changes -- between two consecutive training inputs is at most three. In particular, as the number of neurons of the network grows, the SGD dynamics is captured by the solution of a gradient flow and, at convergence, the distribution of the weights approaches the unique minimizer of a related free energy, which has a Gibbs form. Our key technical contribution consists in the analysis of the estimator resulting from this minimizer: we show that its second derivative vanishes everywhere, except at some specific locations which represent the “knot” points. We also provide empirical evidence that knots at locations distinct from the data points might occur, as predicted by our theory.

JMLR Other · 发表于 2022

Maximum sampled conditional likelihood for informative subsampling

HaiYing Wang, Jae Kwang Kim

摘要

Subsampling is a computationally effective approach to extract information from massive data sets when computing resources are limited. After a subsample is taken from the full data, most available methods use an inverse probability weighted (IPW) objective function to estimate the model parameters. The IPW estimator does not fully utilize the information in the selected subsample. In this paper, we propose to use the maximum sampled conditional likelihood estimator (MSCLE) based on the sampled data. We established the asymptotic normality of the MSCLE and prove that its asymptotic variance covariance matrix is the smallest among a class of asymptotically unbiased estimators, including the IPW estimator. We further discuss the asymptotic results with the L-optimal subsampling probabilities and illustrate the estimation procedure with generalized linear models. Numerical experiments are provided to evaluate the practical performance of the proposed method.

Econometrica Other · 发表于 2022

Maximum Likelihood Estimation in Markov Regime‐Switching Models With Covariate‐Dependent Transition Probabilities

Demian Pouzo, Zacharias Psaradakis, Martin Sola

摘要

This paper considers maximum likelihood (ML) estimation in a large class of models with hidden Markov regimes. We investigate consistency of the ML estimator and local asymptotic normality for the models under general conditions, which allow for autoregressive dynamics in the observable process, Markov regime sequences with covariate‐dependent transition matrices, and possible model misspecification. A Monte Carlo study examines the finite‐sample properties of the ML estimator in correctly specified and misspecified models. An empirical application is also discussed.

JMLR Missing Data & Measurement Error · 发表于 2022

Matrix Completion with Covariate Information and Informative Missingness

Huaqing Jin, Yanyuan Ma, Fei Jiang

摘要

We study the problem of matrix completion when the missingness of the matrix entries is dependent on the unobserved response values themselves and hence the missingness itself is informative. Furthermore, we allow to take into account the covariate information to establish its relation with the response and hence enable prediction. We devise a novel procedure to simultaneously complete the partially observed matrix and assess the covariate effect. Allowing the matrix dimensions as well as the number of covariates to grow ultra-high, under the classic low-rank matrix and sparse covariate effect assumptions, we rigorously establish the statistical guarantee of our procedure and the algorithmic convergence. The method is demonstrated via simulation studies and is used to analyze a Yelp data set and a MovieLens data set.

Econometrica Spatial & Time Series Econometrics & Economics · 发表于 2022

Market Size and Spatial Growth—Evidence From Germany's Post‐War Population Expulsions

Michael Peters

摘要

Virtually all theories of economic growth predict a positive relationship between population size and productivity. In this paper, I study a particular historical episode to provide direct evidence for the empirical relevance of such scale effects. In the aftermath of the Second World War, 8 million ethnic Germans were expelled from their domiciles in Eastern Europe and transferred to West Germany. This inflow increased the German population by almost 20%. Using variation across counties, I show that the settlement of refugees had large and persistent effects on the size of the local population, manufacturing employment, and income per capita. These findings are quantitatively consistent with an idea‐based model of spatial growth if population mobility is subject to frictions and productivity spillovers occur locally. The estimated model implies that the refugee settlement increased aggregate income per capita by about 12% after 25 years and triggered a process of industrialization in rural areas.

Econometrica Econometrics & Economics · 发表于 2022

Market Competition and Political Influence: An Integrated Approach

Steven Callander, Dana Foarta, Takuo Sugaya

摘要

The operation of markets and of politics are in practice deeply intertwined. Political decisions set the rules of the game for market competition and, conversely, market competitors participate in and influence political decisions. We develop an integrated model to capture the circularity between the two domains. We show that a positive feedback loop emerges such that market power begets political power, and political power begets market power, but that this feedback loop is bounded. With too much market power, the balance between politics and markets itself becomes lopsided and this drives a wedge between the interests of a policymaker and the dominant firm. Although such a wedge would seem pro‐competitive, we show how it can exacerbate the static and dynamic inefficiency of market outcomes. More generally, our model demonstrates that intuitions about market competition can be upended when competition is intermediated by a strategic policymaker.

JMLR Networks & Graphs · 发表于 2022

Mappings for Marginal Probabilities with Applications to Models in Statistical Physics

Mehdi Molkaraie

摘要

We present local mappings that relate the marginal probabilities of a global probability mass function represented by its primal normal factor graph to the corresponding marginal probabilities in its dual normal factor graph. The mapping is based on the Fourier transform of the local factors of the models. Details of the mapping are provided for the Ising model, where it is proved that the local extrema of the fixed points are attained at the phase transition of the two-dimensional nearest-neighbor Ising model. The results are further extended to the Potts model, to the clock model, and to Gaussian Markov random fields. By employing the mapping, we can transform simultaneously all the estimated marginal probabilities from the dual domain to the primal domain (and vice versa), which is advantageous if estimating the marginals can be carried out more efficiently in the dual domain. An example of particular significance is the ferromagnetic Ising model in a positive external magnetic field. For this model, there exists a rapidly mixing Markov chain (called the subgraphs--world process) to generate configurations in the dual normal factor graph of the model. Our numerical experiments illustrate that the proposed procedure can provide more accurate estimates of marginal probabilities of a global probability mass function in various settings.

JMLR High-Dimensional · 发表于 2022

Manifold Coordinates with Physical Meaning

Samson J. Koelle, Hanyu Zhang, Marina Meila, Yu-Chia Chen

摘要

Manifold embedding algorithms map high-dimensional data down to coordinates in a much lower-dimensional space. One of the aims of dimension reduction is to find intrinsic coordinates that describe the data manifold. The coordinates returned by the embedding algorithm are abstract, and finding their physical or domain-related meaning is not formalized and often left to domain experts. This paper studies the problem of recovering the meaning of the new low-dimensional representation in an automatic, principled fashion. We propose a method to explain embedding coordinates of a manifold as non-linear compositions of functions from a user-defined dictionary. We show that this problem can be set up as a sparse linear Group Lasso recovery problem, find sufficient recovery conditions, and demonstrate its effectiveness on data.

Econometrica Other · 发表于 2022

Managers and Productivity in the Public Sector

Alessandra Fenizia

摘要

This paper studies the impacts of managers in the administrative public sector using novel Italian administrative data containing an output‐based measure of productivity. Exploiting the rotation of managers across sites, I find that a one standard deviation increase in managerial talent raises office productivity by 10%. These gains are driven primarily by the exit of older workers who retire when more productive managers take over. I use these estimates to evaluate the optimal allocation of managers to offices. I find that assigning better managers to the largest and most productive offices would increase output by at least 6.9%.

Econometrica Econometrics & Economics · 发表于 2022

Making a NARCO: Childhood Exposure to Illegal Labor Markets and Criminal Life Paths

Maria Micaela Sviatschi

摘要

This paper provides evidence that exposure to illegal labor markets during childhood leads to the formation of industry‐specific human capital at an early age, putting children on a criminal life path. Using the timing of U.S. antidrug policies, I show that when the return to illegal activities increases in coca suitable areas in Peru, parents increase the use of child labor for coca farming, putting children on a criminal life path. Using administrative records, I show that affected children are about 30% more likely to be incarcerated for violent and drug‐related crimes as adults. No effect in criminality is found for individuals that grow up working in places where the coca produced goes primarily to the legal sector, suggesting that it is the accumulation of human capital specific to the illegal industry that fosters criminal careers. However, the rollout of a conditional cash transfer program that encourages schooling mitigates the effects of exposure to illegal industries, providing further evidence on the mechanisms.

Econometrica Econometrics & Economics · 发表于 2022

Macro‐Finance Decoupling: Robust Evaluations of Macro Asset Pricing Models

Xu Cheng, Winston Wei Dou, Zhipeng Liao

摘要

This paper shows that robust inference under weak identification is important to the evaluation of many influential macro asset pricing models, including (time‐varying) rare‐disaster risk models and long‐run risk models. Building on recent developments in the conditional inference literature, we provide a novel conditional specification test by simulating the critical value conditional on a sufficient statistic. This sufficient statistic can be intuitively interpreted as a measure capturing the macroeconomic information decoupled from the underlying content of asset pricing theories. Macro‐finance decoupling is an effective way to improve the power of the specification test when asset pricing theories are difficult to refute because of a severe imbalance in the information content about the key model parameters between macroeconomic moment restrictions and asset pricing cross‐equation restrictions. We apply the proposed conditional specification test to the evaluation of a time‐varying rare‐disaster risk model and the construction of robust model uncertainty sets.

JMLR Machine Learning Networks & Graphs Econometrics & Economics · 发表于 2022

Machine Learning on Graphs: A Model and Comprehensive Taxonomy

Ines Chami, Sami Abu-El-Haija, Bryan Perozzi, Christopher Ré, Kevin Murphy

摘要

There has been a surge of recent interest in graph representation learning (GRL). GRL methods have generally fallen into three main categories, based on the availability of labeled data. The first, network embedding, focuses on learning unsupervised representations of relational structure. The second, graph regularized neural networks, leverages graphs to augment neural network losses with a regularization objective for semi-supervised learning. The third, graph neural networks, aims to learn differentiable functions over discrete topologies with arbitrary structure. However, despite the popularity of these areas there has been surprisingly little work on unifying the three paradigms. Here, we aim to bridge the gap between network embedding, graph regularization and graph neural networks. We propose a comprehensive taxonomy of GRL methods, aiming to unify several disparate bodies of work. Specifically, we propose the GraphEDM framework, which generalizes popular algorithms for semi-supervised learning (e.g. GraphSage, GCN, GAT), and unsupervised learning (e.g. DeepWalk, node2vec) of graph representations into a single consistent approach. To illustrate the generality of GraphEDM, we fit over thirty existing methods into this framework. We believe that this unifying view both provides a solid foundation for understanding the intuition behind these methods, and enables future research in the area.

JMLR Other · 发表于 2022

MALTS: Matching After Learning to Stretch

Harsh Parikh, Cynthia Rudin, Alexander Volfovsky

摘要

We introduce a flexible framework that produces high-quality almost-exact matches for causal inference. Most prior work in matching uses ad-hoc distance metrics, often leading to poor quality matches, particularly when there are irrelevant covariates. In this work, we learn an interpretable distance metric for matching, which leads to substantially higher quality matches. The learned distance metric stretches the covariate space according to each covariate's contribution to outcome prediction: this stretching means that mismatches on important covariates carry a larger penalty than mismatches on irrelevant covariates. Our ability to learn flexible distance metrics leads to matches that are interpretable and useful for the estimation of conditional average treatment effects.

JMLR Other · 发表于 2022

Low-rank Tensor Learning with Nonconvex Overlapped Nuclear Norm Regularization

Quanming Yao, Yaqing Wang, Bo Han, James T. Kwok

摘要

Nonconvex regularization has been popularly used in low-rank matrix learning. However, extending it for low-rank tensor learning is still computationally expensive. To address this problem, we develop an efficient solver for use with a nonconvex extension of the overlapped nuclear norm regularizer. Based on the proximal average algorithm, the proposed algorithm can avoid expensive tensor folding/unfolding operations. A special “sparse plus low-rank" structure is maintained throughout the iterations, and allows fast computation of the individual proximal steps. Empirical convergence is further improved with the use of adaptive momentum. We provide convergence guarantees to critical points on smooth losses and also on objectives satisfying the Kurdyka-Lojasiewicz condition. While the optimization problem is nonconvex and nonsmooth, we show that its critical points still have good statistical performance on the tensor completion problem. Experiments on various synthetic and real-world data sets show that the proposed algorithm is efficient in both time and space and more accurate than the existing state-of-the-art.

Econometrica Econometrics & Economics · 发表于 2022

Low Interest Rates, Market Power, and Productivity Growth

Ernest Liu, Atif Mian, Amir Sufi

摘要

This study provides a new theoretical result that a decline in the long‐term interest rate can trigger a stronger investment response by market leaders relative to market followers, thereby leading to more concentrated markets, higher profits, and lower aggregate productivity growth. This strategic effect of lower interest rates on market concentration implies that aggregate productivity growth declines as the interest rate approaches zero. The framework is relevant for antitrust policy in a low interest rate environment, and it provides a unified explanation for rising market concentration and falling productivity growth as interest rates in the economy have fallen to extremely low levels.

Econometrica Other · 发表于 2022

Long‐Run Effects of Dynamically Assigned Treatments: A New Methodology and an Evaluation of Training Effects on Earnings

Gerard J. Berg, Johan Vikström

摘要

We propose and implement a new method to estimate treatment effects in settings where individuals need to be in a certain state (e.g., unemployment) to be eligible for a treatment, treatments may commence at different points in time, and the outcome of interest is realized after the individual left the initial state. An example concerns the effect of training on earnings in subsequent employment. Any evaluation needs to take into account that some of those who are not trained at a certain time in unemployment will leave unemployment before training while others will be trained later. We are interested in effects of the treatment at a certain elapsed duration compared to “no treatment at any subsequent duration.” We prove identification under unconfoundedness and propose inverse probability weighting estimators. A key feature is that weights given to outcome observations of nontreated depend on the remaining time in the initial state. We study effects of a training program for unemployed workers in Sweden. Estimates are positive and sizeable, exceeding those obtained with common static methods. This calls for a reappraisal of training as a tool to bring unemployed back to work.

JMLR Optimization & Learning Theory Machine Learning · 发表于 2022

Logarithmic Regret for Episodic Continuous-Time Linear-Quadratic Reinforcement Learning over a Finite-Time Horizon

Matteo Basei, Xin Guo, Anran Hu, Yufei Zhang

摘要

We study finite-time horizon continuous-time linear-quadratic reinforcement learning problems in an episodic setting, where both the state and control coefficients are unknown to the controller. We first propose a least-squares algorithm based on continuous-time observations and controls, and establish a logarithmic regret bound of magnitude $\mathcal{O}((\ln M)(\ln\ln M) )$, with $M$ being the number of learning episodes. The analysis consists of two components: perturbation analysis, which exploits the regularity and robustness of the associated Riccati differential equation; and parameter estimation error, which relies on sub-exponential properties of continuous-time least-squares estimators. We further propose a practically implementable least-squares algorithm based on discrete-time observations and piecewise constant controls, which achieves similar logarithmic regret with an additional term depending explicitly on the time stepsizes used in the algorithm.

Econometrica Nonparametric & Semiparametric Machine Learning High-Dimensional · 发表于 2022

Locally Robust Semiparametric Estimation

Victor Chernozhukov, Juan Carlos Escanciano, Hidehiko Ichimura, Whitney K. Newey, James M. Robins

摘要

Many economic and causal parameters depend on nonparametric or high dimensional first steps. We give a general construction of locally robust/orthogonal moment functions for GMM, where first steps have no effect, locally, on average moment functions. Using these orthogonal moments reduces model selection and regularization bias, as is important in many applications, especially for machine learning first steps. Also, associated standard errors are robust to misspecification when there is the same number of moment functions as parameters of interest. We use these orthogonal moments and cross‐fitting to construct debiased machine learning estimators of functions of high dimensional conditional quantiles and of dynamic discrete choice parameters with high dimensional state variables. We show that additional first steps needed for the orthogonal moment functions have no effect, globally, on average orthogonal moment functions. We give a general approach to estimating those additional first steps. We characterize double robustness and give a variety of new doubly robust moment functions. We give general and simple regularity conditions for asymptotic theory.

JMLR Latent Variables & Mixtures · 发表于 2022

Linearization and Identification of Multiple-Attractor Dynamical Systems through Laplacian Eigenmaps

Bernardo Fichera, Aude Billard

摘要

Dynamical Systems (DS) are fundamental to the modeling and understanding time evolving phenomena, and have application in physics, biology and control. As determining an analytical description of the dynamics is often difficult, data-driven approaches are preferred for identifying and controlling nonlinear DS with multiple equilibrium points. Identification of such DS has been treated largely as a supervised learning problem. Instead, we focus on an unsupervised learning scenario where we know neither the number nor the type of dynamics. We propose a Graph-based spectral clustering method that takes advantage of a velocity-augmented kernel to connect data points belonging to the same dynamics, while preserving the natural temporal evolution. We study the eigenvectors and eigenvalues of the Graph Laplacian and show that they form a set of orthogonal embedding spaces, one for each sub-dynamics. We prove that there always exist a set of 2-dimensional embedding spaces in which the sub-dynamics are linear and n-dimensional embedding spaces where they are quasi-linear. We compare the clustering performance of our algorithm to Kernel K-Means, Spectral Clustering and Gaussian Mixtures and show that, even when these algorithms are provided with the correct number of sub-dynamics, they fail to cluster them correctly. We learn a diffeomorphism from the Laplacian embedding space to the original space and show that the Laplacian embedding leads to good reconstruction accuracy and a faster training time through an exponential decaying loss compared to the state-of-the-art diffeomorphism-based approaches.

JMLR Econometrics & Economics Genetics & Omics · 发表于 2022

LinCDE: Conditional Density Estimation via Lindsey's Method

Zijun Gao, Trevor Hastie

摘要

Conditional density estimation is a fundamental problem in statistics, with scientific and practical applications in biology, economics, finance and environmental studies, to name a few. In this paper, we propose a conditional density estimator based on gradient boosting and Lindsey's method (LinCDE). LinCDE admits flexible modeling of the density family and can capture distributional characteristics like modality and shape. In particular, when suitably parametrized, LinCDE will produce smooth and non-negative density estimates. Furthermore, like boosted regression trees, LinCDE does automatic feature selection. We demonstrate LinCDE's efficacy through extensive simulations and three real data examples.

JMLR Other · 发表于 2022

Let's Make Block Coordinate Descent Converge Faster: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence

Julie Nutini, Issam Laradji, Mark Schmidt

摘要

Block coordinate descent (BCD) methods are widely used for large-scale numerical optimization because of their cheap iteration costs, low memory requirements, amenability to parallelization, and ability to exploit problem structure. Three main algorithmic choices influence the performance of BCD methods: the block partitioning strategy, the block selection rule, and the block update rule. In this paper we explore all three of these building blocks and propose variations for each that can significantly improve the progress made by each BCD iteration. We (i) propose new greedy block-selection strategies that guarantee more progress per iteration than the Gauss-Southwell rule; (ii) explore practical issues like how to implement the new rules when using "variable" blocks; (iii) explore the use of message-passing to compute matrix or Newton updates efficiently on huge blocks for problems with sparse dependencies between variables; and (iv) consider optimal active manifold identification, which leads to bounds on the "active-set complexity" of BCD methods and leads to superlinear convergence for certain problems with sparse solutions (and in some cases finite termination at an optimal solution). We support all of our findings with numerical results for the classic machine learning problems of least squares, logistic regression, multi-class logistic regression, label propagation, and L1-regularization.

JMLR Optimization & Learning Theory · 发表于 2022

Learning to Optimize: A Primer and A Benchmark

Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang, Wotao Yin

摘要

Learning to optimize (L2O) is an emerging approach that leverages machine learning to develop optimization methods, aiming at reducing the laborious iterations of hand engineering. It automates the design of an optimization method based on its performance on a set of training problems. This data-driven procedure generates methods that can efficiently solve problems similar to those in training. In sharp contrast, the typical and traditional designs of optimization methods are theory-driven, so they obtain performance guarantees over the classes of problems specified by the theory. The difference makes L2O suitable for repeatedly solving a particular optimization problem over a specific distribution of data, while it typically fails on out-of-distribution problems. The practicality of L2O depends on the type of target optimization, the chosen architecture of the method to learn, and the training procedure. This new paradigm has motivated a community of researchers to explore L2O and report their findings. This article is poised to be the first comprehensive survey and benchmark of L2O for continuous optimization. We set up taxonomies, categorize existing works and research directions, present insights, and identify open challenges. We benchmarked many existing L2O approaches on a few representative optimization problems. For reproducible research and fair benchmarking purposes, we released our software implementation and data in the package Open-L2O at https://github.com/VITA-Group/Open-L2O.

JMLR Networks & Graphs · 发表于 2022

Learning linear non-Gaussian directed acyclic graph with diverging number of nodes

Ruixuan Zhao, Xin He, Junhui Wang

摘要

An acyclic model, often depicted as a directed acyclic graph (DAG), has been widely employed to represent directional causal relations among collected nodes. In this article, we propose an efficient method to learn linear non-Gaussian DAG in high dimensional cases, where the noises can be of any continuous non-Gaussian distribution. The proposed method leverages the concept of topological layer to facilitate the DAG learning, and its theoretical justification in terms of exact DAG recovery is also established under mild conditions. Particularly, we show that the topological layers can be exactly reconstructed in a bottom-up fashion, and the parent-child relations among nodes can also be consistently established. The established asymptotic DAG recovery is in sharp contrast to that of many existing learning methods assuming parental faithfulness or ordered noise variances. The advantage of the proposed method is also supported by the numerical comparison against some popular competitors in various simulated examples as well as a real application on the global spread of COVID-19.

JMLR Other · 发表于 2022

Learning from Noisy Pairwise Similarity and Unlabeled Data

Songhua Wu, Tongliang Liu, Bo Han, Jun Yu, Gang Niu, Masashi Sugiyama

摘要

SU classification employs similar (S) data pairs (two examples belong to the same class) and unlabeled (U) data points to build a classifier, which can serve as an alternative to the standard supervised trained classifiers requiring data points with class labels. SU classification is advantageous because in the era of big data, more attention has been paid to data privacy. Datasets with specific class labels are often difficult to obtain in real-world classification applications regarding privacy-sensitive matters, such as politics and religion, which can be a bottleneck in supervised classification. Fortunately, similarity labels do not reveal the explicit information and inherently protect the privacy, e.g., collecting answers to “With whom do you share the same opinion on issue $\mathcal{I}$?" instead of “What is your opinion on issue $\mathcal{I}$?". Nevertheless, SU classification still has an obvious limitation: respondents might answer these questions in a manner that is viewed favorably by others instead of answering truthfully. Therefore, there exist some dissimilar data pairs labeled as similar, which significantly degenerates the performance of SU classification. In this paper, we study how to learn from noisy similar (nS) data pairs and unlabeled (U) data, which is called nSU classification. Specifically, we carefully model the similarity noise and estimate the noise rate by using the mixture proportion estimation technique. Then, a clean classifier can be learned by minimizing a denoised and unbiased classification risk estimator, which only involves the noisy data. Moreover, we further derive a theoretical generalization error bound for the proposed method. Experimental results demonstrate the effectiveness of the proposed algorithm on several benchmark datasets.

JMLR Spatial & Time Series Bayesian Methods Nonparametric & Semiparametric · 发表于 2022

Learning Temporal Evolution of Spatial Dependence with Generalized Spatiotemporal Gaussian Process Models

Shiwei Lan

摘要

A large number of scientific studies involve high-dimensional spatiotemporal data with complicated relationships. In this paper, we focus on a type of space-time interaction named temporal evolution of spatial dependence (TESD), which is a zero time-lag spatiotemporal covariance. For this purpose, we propose a novel Bayesian nonparametric method based on non-stationary spatiotemporal Gaussian process (STGP). The classic STGP has a covariance kernel separable in space and time, failed to characterize TESD. More recent works on non-separable STGP treat location and time together as a joint variable, which is unnecessarily inefficient. We generalize STGP (gSTGP) to introduce time-dependence to the spatial kernel by varying its eigenvalues over time in the Mercer's representation. The resulting non-stationary non-separable covariance model bares a quasi Kronecker sum structure. Finally, a hierarchical Bayesian model for the joint covariance is proposed to allow for full flexibility in learning TESD. A simulation study and a longitudinal neuroimaging analysis on Alzheimer's patients demonstrate that the proposed methodology is (statistically) effective and (computationally) efficient in characterizing TESD. Theoretic properties of gSTGP including posterior contraction (for covariance) are also studied.

JMLR Machine Learning Networks & Graphs · 发表于 2022

Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training

Diego Granziol, Stefan Zohren, Stephen Roberts

摘要

We study the effect of mini-batching on the loss landscape of deep neural networks using spiked, field-dependent random matrix theory. We demonstrate that the magnitude of the extremal values of the batch Hessian are larger than those of the empirical Hessian. We also derive similar results for the Generalised Gauss-Newton matrix approximation of the Hessian. As a consequence of our theorems we derive an analytical expressions for the maximal learning rates as a function of batch size, informing practical training regimens for both stochastic gradient descent (linear scaling) and adaptive algorithms, such as Adam (square root scaling), for smooth, non-convex deep neural networks. Whilst the linear scaling for stochastic gradient descent has been derived under more restrictive conditions, which we generalise, the square root scaling rule for adaptive optimisers is, to our knowledge, completely novel. We validate our claims on the VGG/WideResNet architectures on the CIFAR-100 and ImageNet data sets. Based on our investigations of the sub-sampled Hessian we develop a stochastic Lanczos quadrature based on the fly learning rate and momentum learner, which avoids the need for expensive multiple evaluations for these key hyper-parameters and shows good preliminary results on the Pre-Residual Architecture for CIFAR-100. We further investigate the similarity between the Hessian spectrum of a multi-layer perceptron, trained on Gaussian mixture data, compared to that of deep neural networks trained on natural images. We find striking similarities, with both exhibiting rank degeneracy, a bulk spectrum and outliers to that spectrum. Furthermore, we show that ZCA whitening can remove such outliers early on in training before class separation occurs, but that outliers persist in later training.

JMLR Other · 发表于 2022

Learning Operators with Coupled Attention

Georgios Kissas, Jacob H. Seidman, Leonardo Ferreira Guilhoto, Victor M. Preciado, George J. Pappas, Paris Perdikaris

摘要

Supervised operator learning is an emerging machine learning paradigm with applications to modeling the evolution of spatio-temporal dynamical systems and approximating general black-box relationships between functional data. We propose a novel operator learning method, LOCA (Learning Operators with Coupled Attention), motivated from the recent success of the attention mechanism. In our architecture, the input functions are mapped to a finite set of features which are then averaged with attention weights that depend on the output query locations. By coupling these attention weights together with an integral transform, LOCA is able to explicitly learn correlations in the target output functions, enabling us to approximate nonlinear operators even when the number of output function measurements in the training set is very small. Our formulation is accompanied by rigorous approximation theoretic guarantees on the universal expressiveness of the proposed model. Empirically, we evaluate the performance of LOCA on several operator learning scenarios involving systems governed by ordinary and partial differential equations, as well as a black-box climate prediction problem. Through these scenarios we demonstrate state of the art accuracy, robustness with respect to noisy input data, and a consistently small spread of errors over testing data sets, even for out-of-distribution prediction tasks.

JMLR Other · 发表于 2022

Learning Green's functions associated with time-dependent partial differential equations

Nicolas Boullé, Seick Kim, Tianyi Shi, Alex Townsend

Econometrica Bayesian Methods · 发表于 2022

Learning From Reviews: The Selection Effect and the Speed of Learning

Daron Acemoglu, Ali Makhdoumi, Azarakhsh Malekian, Asuman Ozdaglar

摘要

This paper develops a model of Bayesian learning from online reviews and investigates the conditions for learning the quality of a product and the speed of learning under different rating systems. A rating system provides information about reviews left by previous customers. observe the ratings of a product and decide whether to purchase and review it. We study learning dynamics under two classes of rating systems: full history , where customers see the full history of reviews, and summary statistics , where the platform reports some summary statistics of past reviews. In both cases, learning dynamics are complicated by a selection effect —the types of users who purchase the good, and thus their overall satisfaction and reviews depend on the information available at the time of purchase. We provide conditions for complete learning and characterize and compare its speed under full history and summary statistics. We also show that providing more information does not always lead to faster learning, but strictly finer rating systems do.

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 页