研究园简易版 · 经典书籍 · 期刊文章 · 有原文 · 完整版(可多选筛选、搜索) · 首页
2021 年的期刊文章 · 第 30 页
共 1542 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2021 到 2021。
按期刊、按主题看 →
Biometrics Longitudinal Data · 发表于 2021
Christopher A. German, Janet S. Sinsheimer, Jin Zhou, Hua Zhou
摘要
The availability of vast amounts of longitudinal data from electronic health records (EHRs) and personal wearable devices opens the door to numerous new research questions. In many studies, individual variability of a longitudinal outcome is as important as the mean. Blood pressure fluctuations, glycemic variations, and mood swings are prime examples where it is critical to identify factors that affect the within‐individual variability. We propose a scalable method, within‐subject variance estimator by robust regression (WiSER), for the estimation and inference of the effects of both time‐varying and time‐invariant predictors on within‐subject variance. It is robust against the misspecification of the conditional distribution of responses or the distribution of random effects. It shows similar performance as the correctly specified likelihood methods but is 10 3 ∼ 10 5 times faster. The estimation algorithm scales linearly in the total number of observations, making it applicable to massive longitudinal data sets. The effectiveness of WiSER is evaluated in extensive simulation studies. Its broad applicability is illustrated using the accelerometry data from the Women's Health Study and a clinical trial for longitudinal diabetes care.
Econometrica Econometrics & Economics · 发表于 2021
Raúl Sánchez de la Sierra
摘要
To measure the benefits of formal contract enforcement for society, I create a market with merchants and buyers, in which buyers can choose whether to buy, and whether to pay. A set of multiple “state‐favored” ethnic groups control the state. I experimentally vary whether formal contracts are required and the composition of buyer‐merchant pairs. The design separately identifies the effect of the contracts on the buyers' incentive to pay and on their incentive to buy. I document two ways in which society limits the benefits of contracts. First, contracts reduce buyer cheating, thus increasing merchants' profits, if, and only if, the merchant is state‐favored. Buyers' beliefs suggest that the merchants can enforce the contracts if, and only if, the merchant is state‐favored. Second, holding constant whether the pair is state‐favored, contracts only influence buyer choices when the buyer and the merchant belong to two, different, state‐favored ethnic groups. Buyers' choices and beliefs confirm that, in that case, the contracts are expected to be enforceable, but they have no effect on buyers' choices because reputation already governs the incentives to cheat within groups. The findings temper the view of the state as independent from society, offer a rationale for why contracts are not adopted, and nuance the notion of state weakness.
JMLR Bayesian Methods · 发表于 2021
Purnamrita Sarkar, Y. X. Rachel Wang, Soumendu S. Mukherjee
摘要
Variational approximation has been widely used in large-scale Bayesian inference recently, the simplest kind of which involves imposing a mean field assumption to approximate complicated latent structures. Despite the computational scalability of mean field, theoretical studies of its loss function surface and the convergence behavior of iterative updates for optimizing the loss are far from complete. In this paper, we focus on the problem of community detection for a simple two-class Stochastic Blockmodel (SBM) with equal class sizes. Using batch co-ordinate ascent (BCAVI) for updates, we show different convergence behavior with respect to different initializations. When the parameters are known or estimated within a reasonable range and held fixed, we characterize conditions under which an initialization can converge to the ground truth. On the other hand, when the parameters need to be estimated iteratively, a random initialization will converge to an uninformative local optimum.
Econometrica Spatial & Time Series · 发表于 2021
Rui Da, Dacheng Xiu
摘要
We conduct inference on volatility with noisy high‐frequency data. We assume the observed transaction price follows a continuous‐time Itô‐semimartingale, contaminated by a discrete‐time moving‐average noise process associated with the arrival of trades. We estimate volatility, defined as the quadratic variation of the semimartingale, by maximizing the likelihood of a misspecified moving‐average model, with its order selected based on an information criterion. Our inference is uniformly valid over a large class of noise processes whose magnitude and dependence structure vary with sample size. We show that the convergence rate of our estimator dominates n 1/4 as noise vanishes, and is determined by the selected order of noise dependence when noise is sufficiently small. Our implementation guarantees positive estimates in finite samples.
JMLR Networks & Graphs Optimization & Learning Theory · 发表于 2021
Niladri S. Chatterji, Philip M. Long, Peter L. Bartlett
摘要
We study the training of finite-width two-layer smoothed ReLU networks for binary classification using the logistic loss. We show that gradient descent drives the training loss to zero if the initial loss is small enough. When the data satisfies certain cluster and separation conditions and the network is wide enough, we show that one step of gradient descent reduces the loss sufficiently that the first result applies.
Econometrica Nonparametric & Semiparametric · 发表于 2021
Fatih Guvenen, Fatih Karahan, Serdar Ozkan, Jae Song
摘要
We study individual male earnings dynamics over the life cycle using panel data on millions of U.S. workers. Using nonparametric methods, we first show that the distribution of earnings changes exhibits substantial deviations from lognormality, such as negative skewness and very high kurtosis. Further, the extent of these nonnormalities varies significantly with age and earnings level, peaking around age 50 and between the 70th and 90th percentiles of the earnings distribution. Second, we estimate nonparametric impulse response functions and find important asymmetries: Positive changes for high‐income individuals are quite transitory, whereas negative ones are very persistent; the opposite is true for low‐income individuals. Third, we turn to long‐run outcomes and find substantial heterogeneity in the cumulative growth rates of earnings and the total number of years individuals spend nonemployed between ages 25 and 55. Finally, by targeting these rich sets of moments, we estimate stochastic processes for earnings that range from the simple to the complex. Our preferred specification features normal mixture innovations to both persistent and transitory components and includes state‐dependent long‐term nonemployment shocks with a realization probability that varies with age and earnings.
JMLR Other · 发表于 2021
Licong Lin, Edgar Dobriban
摘要
Modern machine learning methods are often overparametrized, allowing adaptation to the data at a fine level. This can seem puzzling; in the worst case, such models do not need to generalize. This puzzle inspired a great amount of work, arguing when overparametrization reduces test error, in a phenomenon called `double descent'. Recent work aimed to understand in greater depth why overparametrization is helpful for generalization. This lead to discovering the unimodality of variance as a function of the level of parametrization, and to decomposing the variance into that arising from label noise, initialization, and randomness in the training data to understand the sources of the error. In this work we develop a deeper understanding of this area. Specifically, we propose using the analysis of variance (ANOVA) to decompose the variance in the test error in a symmetric way, for studying the generalization performance of certain two-layer linear and non-linear networks. The advantage of the analysis of variance is that it reveals the effects of initialization, label noise, and training data more clearly than prior approaches. Moreover, we also study the monotonicity and unimodality of the variance components. While prior work studied the unimodality of the overall variance, we study the properties of each term in the variance decomposition. One of our key insights is that often, the interaction between training samples and initialization can dominate the variance; surprisingly being larger than their marginal effect. Also, we characterize `phase transitions' where the variance changes from unimodal to monotone. On a technical level, we leverage advanced deterministic equivalent techniques for Haar random matrices, that---to our knowledge---have not yet been used in the area. We verify our results in numerical simulations and on empirical data examples.
JMLR Optimization & Learning Theory · 发表于 2021
Jesus Maria Sanz-Serna, Konstantinos C. Zygalakis
摘要
We present a framework that allows for the non-asymptotic study of the $2$-Wasserstein distance between the invariant distribution of an ergodic stochastic differential equation and the distribution of its numerical approximation in the strongly log-concave case. This allows us to study in a unified way a number of different integrators proposed in the literature for the overdamped and underdamped Langevin dynamics. In addition, we analyze a novel splitting method for the underdamped Langevin dynamics which only requires one gradient evaluation per time step. Under an additional smoothness assumption on a $d$--dimensional strongly log-concave distribution with condition number $\kappa$, the algorithm is shown to produce with an $\mathcal{O}\big(\kappa^{5/4} d^{1/4}\epsilon^{-1/2} \big)$ complexity samples from a distribution that, in Wasserstein distance, is at most $\epsilon>0$ away from the target distribution.
JMLR Optimization & Learning Theory · 发表于 2021
Jason M Altschuler, Enric Boix-Adsera
摘要
Computing Wasserstein barycenters is a fundamental geometric problem with widespread applications in machine learning, statistics, and computer graphics. However, it is unknown whether Wasserstein barycenters can be computed in polynomial time, either exactly or to high precision (i.e., with $\textrm{polylog}(1/\varepsilon)$ runtime dependence). This paper answers these questions in the affirmative for any fixed dimension. Our approach is to solve an exponential-size linear programming formulation by efficiently implementing the corresponding separation oracle using techniques from computational geometry.
Econometrica Econometrics & Economics · 发表于 2021
Matteo Burzoni, Frank Riedel, H. Mete Soner
摘要
We reconsider the microeconomic foundations of financial economics. Motivated by the importance of Knightian uncertainty in markets, we present a model that does not carry any probabilistic structure ex ante, yet is based on a common order. We derive the fundamental equivalence of economic viability of asset prices and absence of arbitrage. We also obtain a modified version of the fundamental theorem of asset pricing using the notion of sublinear pricing measures. Different versions of the efficient market hypothesis are related to the assumptions one is willing to impose on the common order.
Biometrics Survey & Sampling · 发表于 2021
Dewei Wang, Xichen Mou, Yan Liu
摘要
Human biomonitoring involves measuring the accumulation of contaminants in biological specimens (such as blood or urine) to assess individuals' exposure to environmental contamination. Due to the expensive cost of a single assay, the method of pooling has become increasingly common in environmental studies. The implementation of pooling starts by physically mixing specimens into pools, and then measures pooled specimens for the concentration of contaminants. An important task is to reconstruct individual‐level statistical characteristics based on pooled measurements. In this article, we propose to use the varying‐coefficient regression model for individual‐level biomonitoring and provide methods to estimate the varying coefficients based on different types of pooled data. Asymptotic properties of the estimators are presented. We illustrate our methodology via simulation and with application to pooled biomonitoring of a brominated flame retardant provided by the National Health and Nutrition Examination Survey (NHANES).
JMLR Meta-Analysis & Data Integration · 发表于 2021
Jiyuan Tu, Weidong Liu, Xiaojun Mao, Xi Chen
摘要
This paper develops an efficient distributed inference algorithm, which is robust against a moderate fraction of Byzantine nodes, namely arbitrary and possibly adversarial machines in a distributed learning system. In robust statistics, the median-of-means (MOM) has been a popular approach to hedge against Byzantine failures due to its ease of implementation and computational efficiency. However, the MOM estimator has the shortcoming in terms of statistical efficiency. The first main contribution of the paper is to propose a variance reduced median-of-means (VRMOM) estimator, which improves the statistical efficiency over the vanilla MOM estimator and is computationally as efficient as the MOM. Based on the proposed VRMOM estimator, we develop a general distributed inference algorithm that is robust against Byzantine failures. Theoretically, our distributed algorithm achieves a fast convergence rate with only a constant number of rounds of communications. We also provide the asymptotic normality result for the purpose of statistical inference. To the best of our knowledge, this is the first normality result in the setting of Byzantine-robust distributed learning. The simulation results are also presented to illustrate the effectiveness of our method.
JMLR Bayesian Methods · 发表于 2021
Luisa Zintgraf, Sebastian Schulze, Cong Lu, Leo Feng, Maximilian Igl, Kyriacos Shiarlis, Yarin Gal, Katja Hofmann, Shimon Whiteson
摘要
Trading off exploration and exploitation in an unknown environment is key to maximising expected online return during learning. A Bayes-optimal policy, which does so optimally, conditions its actions not only on the environment state but also on the agent's uncertainty about the environment. Computing a Bayes-optimal policy is however intractable for all but the smallest tasks. In this paper, we introduce variational Bayes-Adaptive Deep RL (variBAD), a way to meta-learn approximately Bayes-optimal policies for complex tasks. VariBAD simultaneously meta-learns a variational auto-encoder to perform approximate inference, and a policy that incorporates task uncertainty directly during action selection by conditioning on both the environment state and the approximate belief. In two toy domains, we illustrate how variBAD performs structured online exploration as a function of task uncertainty. We further evaluate variBAD on MuJoCo tasks widely used in meta-RL and show that it achieves higher online return than existing methods. On the recently proposed Meta-World ML1 benchmark, variBAD achieves state of the art results by a large margin, fully solving two out of the three ML1 tasks for the first time.
JMLR Other · 发表于 2021
Zhengze Zhou, Lucas Mentch, Giles Hooker
摘要
As machine learning procedures become an increasingly popular modeling option among applied researchers, there has been a corresponding interest in developing valid tools for understanding their statistical properties and uncertainty. Tree-based ensembles like random forests remain one such popular option for which several important theoretical advances have been made in recent years by drawing upon a connection between their natural subsampled structure and the classical theory of $U$-statistics. Unfortunately, the procedures for estimating predictive variance resulting from these studies are plagued by severe bias and extreme computational overhead. Here, we argue that the root of these problems lies in the use of subsampling without replacement and that with-replacement subsamples, resulting in $V$-statistics, substantially alleviates these problems. We develop a general framework for analyzing the asymptotic behavior of $V$-statistics, demonstrating asymptotic normality under precise regularity conditions and establishing previously unreported connections to $U$-statistics. Importantly, these findings allow us to produce a natural and efficient means of estimating the variance of a conditional expectation, a problem of wide interest across multiple scientific domains that also lies at the heart of uncertainty quantification for supervised learning ensembles.
Econometrica Econometrics & Economics · 发表于 2021
Adrien Auclert, Bence Bardóczy, Matthew Rognlie, Ludwig Straub
摘要
We propose a general and highly efficient method for solving and estimating general equilibrium heterogeneous‐agent models with aggregate shocks in discrete time. Our approach relies on the rapid computation of sequence‐space Jacobians —the derivatives of perfect‐foresight equilibrium mappings between aggregate sequences around the steady state. Our main contribution is a fast algorithm for calculating Jacobians for a large class of heterogeneous‐agent problems. We combine this algorithm with a systematic approach to composing and inverting Jacobians to solve for general equilibrium impulse responses. We obtain a rapid procedure for likelihood‐based estimation and computation of nonlinear perfect‐foresight transitions. We apply our methods to three canonical heterogeneous‐agent models: a neoclassical model, a New Keynesian model with one asset, and a New Keynesian model with two assets.
JMLR Other · 发表于 2021
Fadoua Balabdaoui, Charles R. Doss, Cécile Durot
摘要
We consider so-called univariate unlinked (sometimes “decoupled,” or “shuffled”) regression when the unknown regression curve is monotone. In standard monotone regression, one observes a pair $(X,Y)$ where a response $Y$ is linked to a covariate $X$ through the model $Y= m_0(X) + \epsilon$, with $m_0$ the (unknown) monotone regression function and $\epsilon$ the unobserved error (assumed to be independent of $X$). In the unlinked regression setting one gets only to observe a vector of realizations from both the response $Y$ and from the covariate $X$ where now $Y \stackrel{d}{=} m_0(X) + \epsilon$. There is no (observed) pairing of $X$ and $Y$. Despite this, it is actually still possible to derive a consistent non-parametric estimator of $m_0$ under the assumption of monotonicity of $m_0$ and knowledge of the distribution of the noise $\epsilon$. In this paper, we establish an upper bound on the rate of convergence of such an estimator under minimal assumption on the distribution of the covariate $X$. We discuss extensions to the case in which the distribution of the noise is unknown. We develop a second order algorithm for its computation, and we demonstrate its use on synthetic data. Finally, we apply our method (in a fully data driven way, without knowledge of the error distribution) on longitudinal data from the US Consumer Expenditure Survey.
JMLR Other · 发表于 2021
László Györfi, Roi Weiss
摘要
We study universal consistency and convergence rates of simple nearest-neighbor prototype rules for the problem of multiclass classification in metric spaces. We first show that a novel data-dependent partitioning rule, named Proto-NN, is universally consistent in any metric space that admits a universally consistent rule. Proto-NN is a significant simplification of OptiNet, a recently proposed compression-based algorithm that, to date, was the only algorithm known to be universally consistent in such a general setting. Practically, Proto-NN is simpler to implement and enjoys reduced computational complexity. We then proceed to study convergence rates of the excess error probability. We first obtain rates for the standard $k$-NN rule under a margin condition and a new generalized-Lipschitz condition. The latter is an extension of a recently proposed modified-Lipschitz condition from $\mathbb R^d$ to metric spaces. Similarly to the modified-Lipschitz condition, the new condition avoids any boundness assumptions on the data distribution. While obtaining rates for Proto-NN is left open, we show that a second prototype rule that hybridizes between $k$-NN and Proto-NN achieves the same rates as $k$-NN while enjoying similar computational advantages as Proto-NN. However, as $k$-NN, this hybrid rule is not consistent in general.
JMLR Latent Variables & Mixtures · 发表于 2021
Yunxiao Chen, Zhiliang Ying, Haoran Zhang
摘要
Multidimensional unfolding methods are widely used for visualizing item response data. Such methods project respondents and items simultaneously onto a low-dimensional Euclidian space, in which respondents and items are represented by ideal points, with person-person, item-item, and person-item similarities being captured by the Euclidian distances between the points. In this paper, we study the visualization of multidimensional unfolding from a statistical perspective. We cast multidimensional unfolding into an estimation problem, where the respondent and item ideal points are treated as parameters to be estimated. An estimator is then proposed for the simultaneous estimation of these parameters. Asymptotic theory is provided for the recovery of the ideal points, shedding lights on the validity of model-based visualization. An alternating projected gradient descent algorithm is proposed for the parameter estimation. We provide two illustrative examples, one on users' movie rating and the other on senate roll call voting.
JMLR Machine Learning Networks & Graphs Econometrics & Economics · 发表于 2021
Soon Hoe Lim
摘要
Recurrent neural networks (RNNs) are brain-inspired models widely used in machine learning for analyzing sequential data. The present work is a contribution towards a deeper understanding of how RNNs process input signals using the response theory from nonequilibrium statistical mechanics. For a class of continuous-time stochastic RNNs (SRNNs) driven by an input signal, we derive a Volterra type series representation for their output. This representation is interpretable and disentangles the input signal from the SRNN architecture. The kernels of the series are certain recursively defined correlation functions with respect to the unperturbed dynamics that completely determine the output. Exploiting connections of this representation and its implications to rough paths theory, we identify a universal feature -- the response feature, which turns out to be the signature of tensor product of the input signal and a natural support basis. In particular, we show that SRNNs, with only the weights in the readout layer optimized and the weights in the hidden layer kept fixed and not optimized, can be viewed as kernel machines operating on a reproducing kernel Hilbert space associated with the response feature
JMLR Other · 发表于 2021
Yingfan Wang, Haiyang Huang, Cynthia Rudin, Yaron Shaposhnik
摘要
Dimension reduction (DR) techniques such as t-SNE, UMAP, and TriMap have demonstrated impressive visualization performance on many real-world datasets. One tension that has always faced these methods is the trade-off between preservation of global structure and preservation of local structure: these methods can either handle one or the other, but not both. In this work, our main goal is to understand what aspects of DR methods are important for preserving both local and global structure: it is difficult to design a better method without a true understanding of the choices we make in our algorithms and their empirical impact on the low-dimensional embeddings they produce. Towards the goal of local structure preservation, we provide several useful design principles for DR loss functions based on our new understanding of the mechanisms behind successful DR methods. Towards the goal of global structure preservation, our analysis illuminates that the choice of which components to preserve is important. We leverage these insights to design a new algorithm for DR, called Pairwise Controlled Manifold Approximation Projection (PaCMAP), which preserves both local and global structure. Our work provides several unexpected insights into what design choices both to make and avoid when constructing DR algorithms.
JMLR Optimization & Learning Theory · 发表于 2021
Julian Zimmert, Yevgeny Seldin
摘要
We derive an algorithm that achieves the optimal (within constants) pseudo-regret in both adversarial and stochastic multi-armed bandits without prior knowledge of the regime and time horizon. The algorithm is based on online mirror descent (OMD) with Tsallis entropy regularization with power $\alpha=1/2$ and reduced-variance loss estimators. More generally, we define an adversarial regime with a self-bounding constraint, which includes stochastic regime, stochastically constrained adversarial regime, and stochastic regime with adversarial corruptions as special cases, and show that the algorithm achieves logarithmic regret guarantee in this regime and all of its special cases simultaneously with the optimal regret guarantee in the adversarial regime. The algorithm also achieves adversarial and stochastic optimality in the utility-based dueling bandit setting. We provide empirical evaluation of the algorithm demonstrating that it significantly outperforms UCB1 and EXP3 in stochastic environments. We also provide examples of adversarial environments, where UCB1 and Thompson Sampling exhibit almost linear regret, whereas our algorithm suffers only logarithmic regret. To the best of our knowledge, this is the first example demonstrating vulnerability of Thompson Sampling in adversarial environments. Last but not least, we present a general stochastic analysis and a general adversarial analysis of OMD algorithms with Tsallis entropy regularization for $\alpha\in[0,1]$ and explain the reason why $\alpha=1/2$ works best.
JMLR Networks & Graphs Machine Learning · 发表于 2021
Ron Levie, Wei Huang, Lorenzo Bucci, Michael Bronstein, Gitta Kutyniok
摘要
This paper focuses on spectral graph convolutional neural networks (ConvNets), where filters are defined as elementwise multiplication in the frequency domain of a graph. In machine learning settings where the data set consists of signals defined on many different graphs, the trained ConvNet should generalize to signals on graphs unseen in the training set. It is thus important to transfer ConvNets between graphs. Transferability, which is a certain type of generalization capability, can be loosely defined as follows: if two graphs describe the same phenomenon, then a single filter or ConvNet should have similar repercussions on both graphs. This paper aims at debunking the common misconception that spectral filters are not transferable. We show that if two graphs discretize the same “continuous” space, then a spectral filter or ConvNet has approximately the same repercussion on both graphs. Our analysis is more permissive than the standard analysis. Transferability is typically described as the robustness of the filter to small graph perturbations and re-indexing of the vertices. Our analysis accounts also for large graph perturbations. We prove transferability between graphs that can have completely different dimensions and topologies, only requiring that both graphs discretize the same underlying space in some generic sense.
JMLR Bayesian Methods Networks & Graphs Machine Learning · 发表于 2021
James-A. Goulet, Luong Ha Nguyen, Saeid Amiri
摘要
In this paper, we propose an analytical method for performing tractable approximate Gaussian inference (TAGI) in Bayesian neural networks. The method enables the analytical Gaussian inference of the posterior mean vector and diagonal covariance matrix for weights and biases. The method proposed has a computational complexity of $\mathcal{O}(n)$ with respect to the number of parameters $n$, and the tests performed on regression and classification benchmarks confirm that, for a same network architecture, it matches the performance of existing methods relying on gradient backpropagation.
JMLR Other · 发表于 2021
Zhu Li, Jean-Francois Ton, Dino Oglic, Dino Sejdinovic
摘要
Random Fourier features is a widely used, simple, and effective technique for scaling up kernel methods. The existing theoretical analysis of the approach, however, remains focused on specific learning tasks and typically gives pessimistic bounds which are at odds with the empirical results. We tackle these problems and provide the first unified risk analysis of learning with random Fourier features using the squared error and Lipschitz continuous loss functions. In our bounds, the trade-off between the computational cost and the learning risk convergence rate is problem specific and expressed in terms of the regularization parameter and the number of effective degrees of freedom. We study both the standard random Fourier features method for which we improve the existing bounds on the number of features required to guarantee the corresponding minimax risk convergence rate of kernel ridge regression, as well as a data-dependent modification which samples features proportional to ridge leverage scores and further reduces the required number of features. As ridge leverage scores are expensive to compute, we devise a simple approximation scheme which provably reduces the computational cost without loss of statistical efficiency. Our empirical results illustrate the effectiveness of the proposed scheme relative to the standard random Fourier features method.
JMLR Machine Learning Networks & Graphs Bayesian Methods · 发表于 2021
María Pérez-Ortiz, Omar Rivasplata, John Shawe-Taylor, Csaba Szepesvári
摘要
This paper presents an empirical study regarding training probabilistic neural networks using training objectives derived from PAC-Bayes bounds. In the context of probabilistic neural networks, the output of training is a probability distribution over network weights. We present two training objectives, used here for the first time in connection with training neural networks. These two training objectives are derived from tight PAC-Bayes bounds. We also re-implement a previously used training objective based on a classical PAC-Bayes bound, to compare the properties of the predictors learned using the different training objectives. We compute risk certificates for the learnt predictors, based on part of the data used to learn the predictors. We further experiment with different types of priors on the weights (both data-free and data-dependent priors) and neural network architectures. Our experiments on MNIST and CIFAR-10 show that our training methods produce competitive test set errors and non-vacuous risk bounds with much tighter values than previous results in the literature, showing promise not only to guide the learning algorithm through bounding the risk but also for model selection. These observations suggest that the methods studied here might be good candidates for self-certified learning, in the sense of using the whole data set for learning a predictor and certifying its risk on any unseen data (from the same distribution as the training data) potentially without the need for holding out test data.
JMLR Optimization & Learning Theory · 发表于 2021
Zixin Zhong, Wang Chi Chueng, Vincent Y. F. Tan
摘要
Motivated by the important and urgent need for efficient optimization in online recommender systems, we revisit the cascading bandit model proposed by Kveton et al. (2015a). While Thompson sampling (TS) algorithms have been shown to be empirically superior to Upper Confidence Bound (UCB) algorithms for cascading bandits, theoretical guarantees are only known for the latter. In this paper, we first provide a problem-dependent upper bound on the regret of a TS algorithm with Beta-Bernoulli updates; this upper bound is tighter than a recent derivation under a more general setting by Huyuk and Tekin (2019). Next, we design and analyze another TS algorithm with Gaussian updates, TS-Cascade. TS-Cascade achieves the state-of-the-art problem-independent regret bound for cascading bandits. Complementarily, we consider a linear generalization of the cascading bandit model, which allows efficient learning in large-scale cascading bandit problem instances. We introduce and analyze a TS algorithm, which enjoys a regret bound that depends on the dimension of the linear model but not the number of items. Finally, by using information-theoretic techniques and a judicious construction of cascading bandit instances, we derive a nearly-matching lower bound on the expected regret for the standard model. Our paper establishes the first theoretical guarantees on TS algorithms for a stochastic combinatorial bandit problem model with partial feedback. Numerical experiments demonstrate the superiority of the proposed TS algorithms compared to existing UCB-based ones.
Econometrica Nonparametric & Semiparametric · 发表于 2021
Tetsuya Kaji
摘要
We provide general formulation of weak identification in semiparametric models and an efficiency concept. Weak identification occurs when a parameter is weakly regular, that is, when it is locally homogeneous of degree zero. When this happens, consistent or equivariant estimation is shown to be impossible. We then show that there exists an underlying regular parameter that fully characterizes the weakly regular parameter. While this parameter is not unique, concepts of sufficiency and minimality help pin down a desirable one. If estimation of minimal sufficient underlying parameters is inefficient, it introduces noise in the corresponding estimation of weakly regular parameters, whence we can improve the estimators by local asymptotic Rao–Blackwellization. We call an estimator weakly efficient if it does not admit such improvement. New weakly efficient estimators are presented in linear IV and nonlinear regression models. Simulation of a linear IV model demonstrates how 2SLS and optimal IV estimators are improved.
Econometrica Econometrics & Economics Genetics & Omics · 发表于 2021
Gene M. Grossman, Phillip McCalman, Robert W. Staiger
摘要
What incentives do governments have to negotiate trade agreements that constrain their domestic regulatory policies? We study a model in which firms design products to appeal to local consumer tastes, but their fixed costs increase with the difference between versions of their product destined for different markets. In this setting, firms' profit‐maximizing choices of product attributes are globally optimal in the absence of consumption externalities, but national governments have unilateral incentives to invoke regulatory protectionism to induce firm delocation. An efficient trade agreement requires commitments not to engage in such opportunistic behavior. A rule requiring mutual recognition of standards can be used to achieve efficiency, but one that requires only national treatment falls short. When product attributes confer local consumption externalities, an efficient trade agreement must coordinate the fine details of countries' regulatory policies.
JMLR Optimization & Learning Theory · 发表于 2021
Ryan R. Curtin, Marcus Edel, Rahul Ganesh Prabhu, Suryoday Basak, Zhihao Lou, Conrad Sanderson
摘要
We overview the ensmallen numerical optimization library, which provides a flexible C++ framework for mathematical optimization of user-supplied objective functions. Many types of objective functions are supported, including general, differentiable, separable, constrained, and categorical. A diverse set of pre-built optimizers is provided, including Quasi-Newton optimizers and many variants of Stochastic Gradient Descent. The underlying framework facilitates the implementation of new optimizers. Optimization of an objective function typically requires supplying only one or two C++ functions. Custom behavior can be easily specified via callback functions. Empirical comparisons show that ensmallen outperforms other frameworks while providing more functionality. The library is available at https://ensmallen.org and is distributed under the permissive BSD license.
Econometrica Nonparametric & Semiparametric · 发表于 2021
Eben Lazarus, Daniel J. Lewis, James H. Stock
摘要
Heteroskedasticity‐ and autocorrelation‐robust (HAR) inference in time series regression typically involves kernel estimation of the long‐run variance. Conventional wisdom holds that, for a given kernel, the choice of truncation parameter trades off a test's null rejection rate and power, and that this tradeoff differs across kernels. We formalize this intuition: using higher‐order expansions, we provide a unified size‐power frontier for both kernel and weighted orthonormal series tests using nonstandard “fixed‐b” critical values. We also provide a frontier for the subset of these tests for which the fixed‐bdistribution istorF. These frontiers are respectively achieved by the QS kernel and equal‐weighted periodogram. The frontiers have simple closed‐form expressions, which show that the price paid for restricting attention to tests withtandFcritical values is small. The frontiers are derived for the Gaussian multivariate location model, but simulations suggest the qualitative findings extend to stochastic regressors.
JMLR Bayesian Methods Machine Learning Networks & Graphs · 发表于 2021
Takuo Matsubara, Chris J. Oates, François-Xavier Briol
摘要
Bayesian neural networks attempt to combine the strong predictive performance of neural networks with formal quantification of uncertainty associated with the predictive output in the Bayesian framework. However, it remains unclear how to endow the parameters of the network with a prior distribution that is meaningful when lifted into the output space of the network. A possible solution is proposed that enables the user to posit an appropriate Gaussian process covariance function for the task at hand. Our approach constructs a prior distribution for the parameters of the network, called a ridgelet prior, that approximates the posited Gaussian process in the output space of the network. In contrast to existing work on the connection between neural networks and Gaussian processes, our analysis is non-asymptotic, with finite sample-size error bounds provided. This establishes the universality property that a Bayesian neural network can approximate any Gaussian process whose covariance function is sufficiently regular. Our experimental assessment is limited to a proof-of-concept, where we demonstrate that the ridgelet prior can out-perform an unstructured prior on regression problems for which a suitable Gaussian process prior can be provided.
Econometrica Other · 发表于 2021
David K. Levine
摘要
Few want to do business with a partner who has a bad reputation. Consequently, once a bad reputation is established, it can be difficult to get rid of. This leads on the one hand to the intuitive idea that a good reputation is easy to lose and hard to gain. On the other hand, it can lead to a strong form of history dependence in which a single beneficial or adverse event can cast a shadow over a very long period of time. It gives rise to a reputational trap where an agent rationally chooses not to invest in a good reputation because the chances others will find out is too low. Nevertheless, the same agent with a good reputation will make every effort to maintain it. Here, a simple reputational model is constructed and the conditions for there to be a unique equilibrium that constitutes a reputation trap are characterized.
Econometrica Econometrics & Economics · 发表于 2021
Debopam Bhattacharya
摘要
An important goal of empirical demand analysis is choice and welfare prediction on counterfactual budget sets arising from potential policy interventions. Such predictions are more credible when made without arbitrary functional‐form/distributional assumptions, and instead based solely on economic rationality, that is, that choice is consistent with utility maximization by a heterogeneous population. This paper investigates nonparametric economic rationality in the empirically important context of binary choice. We show that under general unobserved heterogeneity, economic rationality is equivalent to a pair of Slutsky‐like shape restrictions on choice‐probability functions. The forms of these restrictions differ from Slutsky inequalities for continuous goods. Unlike McFadden–Richter's stochastic revealed preference, our shape restrictions (a) are global, that is, their forms do not depend on which and how many budget sets are observed, (b) are closed form, hence easy to impose on parametric/semi/nonparametric models in practical applications, and (c) provide computationally simple, theory‐consistent bounds on demand and welfare predictions on counterfactual budge sets.
JMLR Other · 发表于 2021
Carlos A. Gomez-Uribe, Brian Karrer
摘要
Motivated by the needs of online large-scale recommender systems, we specialize the decoupled extended Kalman filter to factorization models, including factorization machines, matrix and tensor factorization, and illustrate the effectiveness of the approach through numerical experiments on synthetic and on real-world data. Online learning of model parameters through the decoupled extended Kalman filter makes factorization models more broadly useful by (i) allowing for more flexible observations through the entire exponential family, (ii) modeling parameter drift, and (iii) producing parameter uncertainty estimates that can enable explore/exploit and other applications. We use a different parameter dynamics than the standard decoupled extended Kalman filter, allowing parameter drift while encouraging reasonable values. We also present an alternate derivation of the extended Kalman filter and decoupled extended Kalman filter that highlights the role of the Fisher information matrix in the extended Kalman filter.
JMLR Nonparametric & Semiparametric · 发表于 2021
Lasse Petersen, Niels Richard Hansen
摘要
The partial copula provides a method for describing the dependence between two random variables $X$ and $Y$ conditional on a third random vector $Z$ in terms of nonparametric residuals $U_1$ and $U_2$. This paper develops a nonparametric test for conditional independence by combining the partial copula with a quantile regression based method for estimating the nonparametric residuals. We consider a test statistic based on generalized correlation between $U_1$ and $U_2$ and derive its large sample properties under consistency assumptions on the quantile regression procedure. We demonstrate through a simulation study that the resulting test is sound under complicated data generating distributions. Moreover, in the examples considered the test is competitive to other state-of-the-art conditional independence tests in terms of level and power, and it has superior power in cases with conditional variance heterogeneity of $X$ and $Y$ given $Z$.
JMLR Machine Learning Meta-Analysis & Data Integration · 发表于 2021
Paweł Rościszewski, Michał Martyniak, Filip Schodowski
摘要
TensorHive is a tool for organizing work of research and engineering teams that use servers with GPUs for machine learning workloads. In a comprehensive web interface, it supports reservation of GPUs for exclusive usage, hardware monitoring, as well as configuring, executing and queuing distributed computational jobs. Focusing on easy installation and simple configuration, the tool automatically detects the available computing resources and monitors their utilization. Reservations granted on the basis of flexible access control settings are protected by pluggable violation hooks. The job execution module includes auto-configuration templates for distributed neural network training jobs in frameworks such as TensorFlow and PyTorch. Documentation, source code, usage examples and issue tracking are available at the project page: https://github.com/roscisz/TensorHive/
Econometrica Other · 发表于 2021
Ben Weidmann, David J. Deming
摘要
Most jobs require teamwork. Are some people good team players? In this paper, we design and test a new method for identifying individual contributions to team production. We randomly assign people to multiple teams and predict team performance based on previously assessed individual skills. Some people consistently cause their team to exceed its predicted performance. We call these individuals “team players.” Team players score significantly higher on a well‐established measure of social intelligence, but do not differ across a variety of other dimensions, including IQ, personality, education, and gender. Social skills—defined as a single latent factor that combines social intelligence scores with the team player effect—improve team performance about as much as IQ. We find suggestive evidence that team players increase effort among teammates.
Econometrica Econometrics & Economics · 发表于 2021
Mohamed Saleh, Jean Tirole
摘要
A ruler who does not identify with a social group, whether on religious, ethnic, cultural, or socioeconomic grounds, is confronted with a trade‐off between taking advantage of the out‐group population's eagerness to maintain its identity and inducing it to “comply” (conversion, quitting, exodus, or any other way to accommodate the ruler's own identity). This paper first nests economists' extraction model, in which rulers are revenue‐maximizers, within a more general identity‐based model, in which rulers care also about inducing people to lose their identity, both in a static and an evolving environment. This paper then constructs novel data sources to test the implications of both models in the context of Egypt's conversion to Islam between 641 and 1170. The evidence supports the identity‐based model.
Econometrica Missing Data & Measurement Error · 发表于 2021
Bradley T. Shapiro, Günter J. Hitsch, Anna E. Tuchman
摘要
We estimate the distribution of television advertising elasticities and the distribution of the advertising return on investment (ROI) for a large number of products in many categories. Our results reveal substantially smaller advertising elasticities compared to the results documented in the literature, as well as a sizable percentage of statistically insignificant or negative estimates. The results are robust to functional form assumptions and are not driven by insufficient statistical power or measurement error. The ROI analysis shows negative ROIs at the margin for more than 80% of brands, implying over‐investment in advertising by most firms. Further, the overall ROI of the observed advertising schedule is only positive for one third of all brands.
JMLR Latent Variables & Mixtures · 发表于 2021
Weiwei Li, Jan Hannig, Sayan Mukherjee
摘要
The problem of dimension reduction is of increasing importance in modern data analysis. In this paper, we consider modeling the collection of points in a high dimensional space as a union of low dimensional subspaces. In particular we propose a highly scalable sampling based algorithm that clusters the entire data via first spectral clustering of a small random sample followed by classifying or labeling the remaining out-of-sample points. The key idea is that this random subset borrows information across the entire dataset and that the problem of clustering points can be replaced with the more efficient problem of "clustering sub-clusters". We provide theoretical guarantees for our procedure. The numerical results indicate that for large datasets the proposed algorithm outperforms other state-of-the-art subspace clustering algorithms with respect to accuracy and speed.