2017 年的期刊文章 · 第 28 页

共 1233 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2017 到 2017。

按期刊、按主题看 →

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 页

Econometrica Econometrics & Economics Nonparametric & Semiparametric · 发表于 2017

Quantile Selection Models With an Application to Understanding Changes in Wage Inequality

Manuel Arellano, Stéphane Bonhomme

摘要

We propose a method to correct for sample selection in quantile regression models.Selection is modelled via the cumulative distribution function, or copula, of the percentile error in the outcome equation and the error in the participation decision.Copula parameters are estimated by minimizing a method-of-moments criterion.Given these parameter estimates, the percentile levels of the outcome are re-adjusted to correct for selection, and quantile parameters are estimated by minimizing a rotated "check" function.We apply the method to correct wage percentiles for selection into employment, using data for the UK for the period 1978-2000.We also extend the method to account for the presence of equilibrium effects when performing counterfactual exercises. *(abstract via OpenAlex)*

JMLR Other · 发表于 2017

Quantifying the Informativeness of Similarity Measurements

Austin J. Brockmeier, Tingting Mu, Sophia Ananiadou, John Y. Goulermas

摘要

In this paper, we describe an unsupervised measure for quantifying the 'informativeness' of correlation matrices formed from the pairwise similarities or relationships among data instances. The measure quantifies the heterogeneity of the correlations and is defined as the distance between a correlation matrix and the nearest correlation matrix with constant off-diagonal entries. This non-parametric notion generalizes existing test statistics for equality of correlation coefficients by allowing for alternative distance metrics, such as the Bures and other distances from quantum information theory. For several distance and dissimilarity metrics, we derive closed-form expressions of informativeness, which can be applied as objective functions for machine learning applications. Empirically, we demonstrate that informativeness is a useful criterion for selecting kernel parameters, choosing the dimension for kernel-based nonlinear dimensionality reduction, and identifying structured graphs. We also consider the problem of finding a maximally informative correlation matrix around a target matrix, and explore parameterizing the optimization in terms of the coordinates of the sample or through a lower-dimensional embedding. In the latter case, we find that maximizing the Bures-based informativeness measure, which is maximal for centered rank-1 correlation matrices, is equivalent to minimizing a specific matrix norm, and present an algorithm to solve the minimization problem using the norm's proximal operator. The proposed correlation denoising algorithm consistently improves spectral clustering. Overall, we find informativeness to be a novel and useful criterion for identifying non-trivial correlation structure.

Econometrica Other · 发表于 2017

Progressive Learning

Avidit Acharya, Juan Ortner

摘要

We study a dynamic principal–agent relationship with adverse selection and limited commitment. We show that when the relationship is subject to productivity shocks, the principal may be able to improve her value over time by progressively learning the agent's private information. She may even achieve her first‐best payoff in the long run. The relationship may also exhibit path dependence, with early shocks determining the principal's long‐run value. These findings contrast sharply with the results of the ratchet effect literature, in which the principal persistently obtains low payoffs, giving up substantial informational rents to the agent. *(abstract via OpenAlex)*

Econometrica High-Dimensional Causal Inference · 发表于 2017

Program Evaluation and Causal Inference With High-Dimensional Data

A. Belloni, V. Chernozhukov, I. Fern�ndez-Val, C. Hansen

摘要

The accepted manuscript version (last revised 5 Jan 2018 (v8)) has 118 pages, 3 tables, 11 figures, and includes supplementary appendix. This version corrects some typos in Example 2 of the published version. This supplement contains 11 appendices with additional results and some omitted proofs. Appendices F-J include additional results for Sections 2-7, respectively. Appendix K gathers auxiliary results on algebra of covering entropies. Appendices L and M contain the proofs of Sections 4 and 5 omitted from the main text. Appendix N contains the proofs of Sections 6 omitted from the main text, together with the proofs of the additional results for Section 6 in Appendix I. Appendix O reports the results of a simulation experiment. *(abstract via OpenAlex)*

JMLR Optimization & Learning Theory · 发表于 2017

Probabilistic Line Searches for Stochastic Optimization

Maren Mahsereci, Philipp Hennig

摘要

In deterministic optimization, line searches are a standard tool ensuring stability and efficiency. Where only stochastic gradients are available, no direct equivalent has so far been formulated, because uncertain gradients do not allow for a strict sequence of decisions collapsing the search space. We construct a probabilistic line search by combining the structure of existing deterministic methods with notions from Bayesian optimization. Our method retains a Gaussian process surrogate of the univariate optimization objective, and uses a probabilistic belief over the Wolfe conditions to monitor the descent. The algorithm has very low computational cost, and no user- controlled parameters. Experiments show that it effectively removes the need to define a learning rate for stochastic gradient descent.

JMLR Other · 发表于 2017

Preference-based Teaching

Ziyuan Gao, Christoph Ries, Hans U. Simon, Sandra Zilles

摘要

We introduce a new model of teaching named preference-based teaching and a corresponding complexity parameter---the preference-based teaching dimension (PBTD)---representing the worst-case number of examples needed to teach any concept in a given concept class. Although the PBTD coincides with the well- known recursive teaching dimension (RTD) on finite classes, it is radically different on infinite ones: the RTD becomes infinite already for trivial infinite classes (such as half- intervals) whereas the PBTD evaluates to reasonably small values for a wide collection of infinite classes including classes consisting of so-called closed sets w.r.t. a given closure operator, including various classes related to linear sets over $\mathbb{N}_0$ (whose RTD had been studied quite recently) and including the class of Euclidean half-spaces. On top of presenting these concrete results, we provide the reader with a theoretical framework (of a combinatorial flavor) which helps to derive bounds on the PBTD.

Econometrica Econometrics & Economics · 发表于 2017

Political Economy of Redistribution

Daniel Diermeier, Georgy Egorov, Konstantin Sonin

摘要

It is often argued that additional constraints on redistribution such as granting veto power to more players in the society makes property better protected from expropriation. We use a legislative bargaining-type model to demonstrate that this intuition may be ‡flawed. Increasing the number of veto players or raising the supermajority requirement for redistribution may reduce protection on the equilibrium path. The reason is the existence of two distinct mechanisms of property rights protection. One is formal constraints that allow individuals or groups to block any redistribution that is not in their favor. The other occurs in equilibrium where players without such powers protect each other from redistribution. Players without formal veto power anticipate that the expropriation of other similar players will ultimately hurt them and thus combine their infl‡uence to prevent redistributions. In a stable allocation, the society exhibits a “"class" ”structure with class members having equal wealth, and strategically protecting each other from redistribution. *(abstract via OpenAlex)*

JMLR Bayesian Methods Latent Variables & Mixtures · 发表于 2017

Poisson Random Fields for Dynamic Feature Models

Valerio Perrone, Paul A. Jenkins, Dario Spanò, Yee Whye Teh

摘要

We present the Wright-Fisher Indian buffet process (WF- IBP), a probabilistic model for time-dependent data assumed to have been generated by an unknown number of latent features. This model is suitable as a prior in Bayesian nonparametric feature allocation models in which the features underlying the observed data exhibit a dependency structure over time. More specifically, we establish a new framework for generating dependent Indian buffet processes, where the Poisson random field model from population genetics is used as a way of constructing dependent beta processes. Inference in the model is complex, and we describe a sophisticated Markov Chain Monte Carlo algorithm for exact posterior simulation. We apply our construction to develop a nonparametric focused topic model for collections of time-stamped text documents and test it on the full corpus of NIPS papers published from 1987 to 2015.

Econometrica Econometrics & Economics · 发表于 2017

Persuasion of a Privately Informed Receiver

Anton Kolotilin, Tymofiy Mylovanov, Andriy Zapechelnyuk, Ming Li

摘要

Kolotilin acknowledges financial support from the Australian Research Council. Zapechelnyuk acknowledges financial support from the Economic and Social Research Council (grant no. ES/N01829X/1) *(abstract via OpenAlex)*

JMLR Other · 发表于 2017

Persistence Images: A Stable Vector Representation of Persistent Homology

Henry Adams, Tegan Emerson, Michael Kirby, Rachel Neville, Chris Peterson, Patrick Shipman, Sofya Chepushtanova, Eric Hanson, Francis Motta, Lori Ziegelmeier

摘要

Many data sets can be viewed as a noisy sampling of an underlying space, and tools from topological data analysis can characterize this structure for the purpose of knowledge discovery. One such tool is persistent homology, which provides a multiscale description of the homological features within a data set. A useful representation of this homological information is a persistence diagram (PD). Efforts have been made to map PDs into spaces with additional structure valuable to machine learning tasks. We convert a PD to a finite- dimensional vector representation which we call a persistence image (PI), and prove the stability of this transformation with respect to small perturbations in the inputs. The discriminatory power of PIs is compared against existing methods, showing significant performance gains. We explore the use of PIs with vector-based machine learning tools, such as linear sparse support vector machines, which identify features containing discriminating topological information. Finally, high accuracy inference of parameter values from the dynamic output of a discrete dynamical system (the linked twist map) and a partial differential equation (the anisotropic Kuramoto-Sivashinsky equation) provide a novel application of the discriminatory power of PIs.

JMLR Other · 发表于 2017

Perishability of Data: Dynamic Pricing under Varying-Coefficient Models

Adel Javanmard

摘要

We consider a firm that sells a large number of products to its customers in an online fashion. Each product is described by a high dimensional feature vector, and the market value of a product is assumed to be linear in the values of its features. Parameters of the valuation model are unknown and can change over time. The firm sequentially observes a product's features and can use the historical sales data (binary sale/no sale feedbacks) to set the price of current product, with the objective of maximizing the collected revenue. We measure the performance of a dynamic pricing policy via regret, which is the expected revenue loss compared to a clairvoyant that knows the sequence of model parameters in advance.

Econometrica Econometrics & Economics · 发表于 2017

Perfect Competition in Markets With Adverse Selection

Eduardo M. Azevedo, Daniel Gottlieb

摘要

This paper proposes a perfectly competitive model of a market with adverse selection. Prices are determined by zero-profit conditions, and the set of traded contracts is determined by free entry. Crucially for applications, contract characteristics are endogenously determined, consumers may have multiple dimensions of private information, and an equilibrium always exists. Equilibrium corresponds to the limit of a differentiated products Bertrand game. We apply the model to establish theoretical results on the equilibrium effects of mandates. Mandates can increase efficiency but have unintended consequences. With adverse selection, an insurance mandate reduces the price of low-coverage policies, which necessarily has indirect effects such as increasing adverse selection on the intensive margin and causing some consumers to purchase less coverage. *(abstract via OpenAlex)*

JMLR Bayesian Methods Latent Variables & Mixtures · 发表于 2017

Particle Gibbs Split-Merge Sampling for Bayesian Inference in Mixture Models

Alexandre Bouchard-Côté, Arnaud Doucet, Andrew Roth

摘要

This paper presents an original Markov chain Monte Carlo method to sample from the posterior distribution of conjugate mixture models. This algorithm relies on a flexible split-merge procedure built using the particle Gibbs sampler introduced in Andrieu et al. (2009, 2010). The resulting so-called Particle Gibbs Split-Merge sampler does not require the computation of a complex acceptance ratio and can be implemented using existing sequential Monte Carlo libraries. We investigate its performance experimentally on synthetic problems as well as on geolocation data. Our results show that for a given computational budget, the Particle Gibbs Split-Merge sampler empirically outperforms existing split merge methods. The code and instructions allowing to reproduce the experiments is available at github.com/aroth85/pgsm.

Econometrica Other · 发表于 2017

Partial Ambiguity

Soo Hong Chew, Bin Miao, Songfa Zhong

摘要

We extend Ellsberg's two-urn paradox and propose three symmetric forms of partial ambiguity by limiting the possible compositions in a deck of 100 red and black cards in three ways. Interval ambiguity involves a symmetric range of 50 − n to 50 + n red cards. Complementarily, disjoint ambiguity arises from two nonintersecting intervals of 0 to n and 100 − n to 100 red cards. Two-point ambiguity involves n or 100 − n red cards. We investigate experimentally attitudes towards partial ambiguity and the corresponding compound lotteries in which the possible compositions are drawn with equal objective probabilities. This yields three key findings: distinct attitudes towards the three forms of partial ambiguity, significant association across attitudes towards partial ambiguity and compound risk, and source preference between two-point ambiguity and two-point compound risk. Our findings help discriminate among models of ambiguity in the literature. *(abstract via OpenAlex)*

Econometrica Econometrics & Economics · 发表于 2017

Parenting With Style: Altruism and Paternalism in Intergenerational Preference Transmission

Matthias Doepke, Fabrizio Zilibotti

摘要

We develop a theory of intergenerational transmission of preferences that rationalizes the choice between alternative parenting styles (as set out in Baumrind1967). ParentsmaximizeanobjectivefunctionthatcombinesBeckerian altruism and paternalism towards children. They can affect their children’s choices via two channels: either by influencing children’s preferences or by imposing direct restrictions on their choice sets. Different parenting styles (authoritarian, authoritative, and permissive) emerge as equilibrium outcomes, and are affected both by parental preferences and by the socioeconomic environment. Parenting style, in turn, feeds back into the children’s welfare and economic success. The theory is consistent with the decline of authoritarian parenting observed in industrialized countries, and with the greater prevalence of more permissive parenting in countries characterized by low inequality. *(abstract via OpenAlex)*

JMLR Other · 发表于 2017

Parallel Symmetric Class Expression Learning

An C. Tran, Jens Dietrich, Hans W. Guesgen, Stephen Marsland

摘要

In machine learning, one often encounters data sets where a general pattern is violated by a relatively small number of exceptions (for example, a rule that says that all birds can fly is violated by examples such as penguins). This complicates the concept learning process and may lead to the rejection of some simple and expressive rules that cover many cases. In this paper we present an approach to this problem in description logic learning by computing partial descriptions (which are not necessarily entirely complete) of both positive and negative examples and combining them. Our Symmetric Parallel Class Expression Learning approach enables the generation of general rules with exception patterns included. We demonstrate that this algorithm provides significantly better results (in terms of metrics such as accuracy, search space covered, and learning time) than standard approaches on some typical data sets. Further, the approach has the added benefit that it can be parallelised relatively simply, leading to much faster exploration of the search tree on modern computers.

JMLR Other · 发表于 2017

POMDPs.jl: A Framework for Sequential Decision Making under Uncertainty

Maxim Egorov, Zachary N. Sunberg, Edward Balaban, Tim A. Wheeler, Jayesh K. Gupta, Mykel J. Kochenderfer

摘要

POMDPs.jl is an open-source framework for solving Markov decision processes (MDPs) and partially observable MDPs (POMDPs). POMDPs.jl allows users to specify sequential decision making problems with minimal effort without sacrificing the expressive nature of POMDPs, making this framework viable for both educational and research purposes. It is written in the Julia language to allow flexible prototyping and large-scale computation that leverages the high-performance nature of the language. The associated JuliaPOMDP community also provides a number of state-of-the-art MDP and POMDP solvers and a rich library of support tools to help with implementing new solvers and evaluating the solution results. The most recent version of POMDPs.jl, the related packages, and documentation can be found at github.com/ JuliaPOMDP/POMDPs.jl.

JMLR Optimization & Learning Theory · 发表于 2017

Optimal Rates for Multi-pass Stochastic Gradient Methods

Junhong Lin, Lorenzo Rosasco

摘要

We analyze the learning properties of the stochastic gradient method when multiple passes over the data and mini-batches are allowed. We study how regularization properties are controlled by the step-size, the number of passes and the mini-batch size. In particular, we consider the square loss and show that for a universal step-size choice, the number of passes acts as a regularization parameter, and optimal finite sample bounds can be achieved by early-stopping. Moreover, we show that larger step-sizes are allowed when considering mini-batches. Our analysis is based on a unifying approach, encompassing both batch and stochastic gradient methods as special cases. As a byproduct, we derive optimal convergence results for batch gradient methods (even in the non-attainable cases).

JMLR Other · 发表于 2017

Optimal Dictionary for Least Squares Representation

Mohammed Rayyan Sheriff, Debasish Chatterjee

摘要

Dictionaries are collections of vectors used for the representation of a class of vectors in Euclidean spaces. Recent research on optimal dictionaries is focused on constructing dictionaries that offer sparse representations, i.e., $\ell_0$-optimal representations. Here we consider the problem of finding optimal dictionaries with which representations of a given class of vectors is optimal in an $\ell_2$-sense: optimality of representation is defined as attaining the minimal average $\ell_2$-norm of the coefficients used to represent the vectors in the given class. With the help of recent results on rank-1 decompositions of symmetric positive semidefinite matrices, we provide an explicit description of $\ell_2$-optimal dictionaries as well as their algorithmic constructions in polynomial time.

JMLR Bayesian Methods Latent Variables & Mixtures · 发表于 2017

Online but Accurate Inference for Latent Variable Models with Local Gibbs Sampling

Christophe Dupuy, Francis Bach

摘要

We study parameter inference in large-scale latent variable models. We first propose a unified treatment of online inference for latent variable models from a non-canonical exponential family, and draw explicit links between several previously proposed frequentist or Bayesian methods. We then propose a novel inference method for the frequentist estimation of parameters, that adapts MCMC methods to online inference of latent variable models with the proper use of local Gibbs sampling. Then, for latent Dirichlet allocation,we provide an extensive set of experiments and comparisons with existing work, where our new approach outperforms all previously proposed methods. In particular, using Gibbs sampling for latent variable inference is superior to variational inference in terms of test log-likelihoods. Moreover, Bayesian inference through variational methods perform poorly, sometimes leading to worse fits with latent variables of higher dimensionality.

JMLR Other · 发表于 2017

Online Learning to Rank with Top-k Feedback

Sougata Chaudhuri, Ambuj Tewari

摘要

We consider two settings of online learning to rank where feedback is restricted to top ranked items. The problem is cast as an online game between a learner and sequence of users, over $T$ rounds. In both settings, the learners objective is to present ranked list of items to the users. The learner's performance is judged on the entire ranked list and true relevances of the items. However, the learner receives highly restricted feedback at end of each round, in form of relevances of only the top $k$ ranked items, where $k \ll m$. The first setting is non-contextual, where the list of items to be ranked is fixed. The second setting is contextual, where lists of items vary, in form of traditional query-document lists. No stochastic assumption is made on the generation process of relevances of items and contexts. We provide efficient ranking strategies for both the settings. The strategies achieve $O(T^{2/3})$ regret, where regret is based on popular ranking measures in first setting and ranking surrogates in second setting. We also provide impossibility results for certain ranking measures and a certain class of surrogates, when feedback is restricted to the top ranked item, i.e. $k=1$. We empirically demonstrate the performance of our algorithms on simulated and real world data sets.

JMLR Bayesian Methods · 发表于 2017

Online Bayesian Passive-Aggressive Learning

Tianlin Shi, Jun Zhu

摘要

We present online Bayesian Passive-Aggressive (BayesPA) learning, a generic online learning framework for hierarchical Bayesian models with max-margin posterior regularization. We show that BayesPA subsumes the standard online Passive- Aggressive (PA) learning and extends naturally to incorporate latent variables for both parametric and nonparametric Bayesian inference, therefore providing great flexibility for explorative analysis. As an important example, we apply BayesPA to topic modeling and derive efficient online learning algorithms for max-margin topic models. We further develop nonparametric BayesPA topic models to infer the unknown number of topics in an online manner. Experimental results on 20newsgroups and a large Wikipedia multi-label dataset (with 1.1 millions of training documents and 0.9 million of unique terms in the vocabulary) show that our approaches significantly improve time efficiency while achieving comparable accuracy with the corresponding batch algorithms.

JMLR Networks & Graphs Missing Data & Measurement Error · 发表于 2017

On the Propagation of Low-Rate Measurement Error to Subgraph Counts in Large Networks

Prakash Balachandran, Eric D. Kolaczyk, Weston D. Viles

摘要

Our work in this paper is inspired by a statistical observation that is both elementary and broadly relevant to network analysis in practice---that the uncertainty in approximating some true graph $G=(V,E)$ by some estimated graph $\hat{G}=(V,\hat{E})$ manifests as errors in our knowledge of the presence/absence of edges between vertex pairs, which must necessarily propagate to any estimates of network summaries $\eta(G)$ we seek. Motivated by the common practice of using plug-in estimates $\eta(\hat{G})$ as proxies for $\eta(G)$, our focus is on the problem of characterizing the distribution of the discrepancy $D=\eta(\hat{G}) - \eta(G)$, in the case where $\eta(\cdot)$ is a subgraph count. Specifically, we study the fundamental case where the statistic of interest is $|E|$, the number of edges in $G$. Our primary contribution in this paper is to show that in the empirically relevant setting of large graphs with low-rate measurement errors, the distribution of $D_E=|\hat{E}| - |E|$ is well-characterized by a Skellam distribution, when the errors are independent or weakly dependent. Under an assumption of independent errors, we are able to further show conditions under which this characterization is strictly better than that of an appropriate normal distribution. These results derive from our formulation of a general result, quantifying the accuracy with which the difference of two sums of dependent Bernoulli random variables may be approximated by the difference of two independent Poisson random variables, i.e., by a Skellam distribution. This general result is developed through the use of Stein's method, and may be of some general interest. We finish with a discussion of possible extension of our work to subgraph counts $\eta(G)$ of higher order.

JMLR Nonparametric & Semiparametric · 发表于 2017

On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions

Francis Bach

摘要

We show that kernel-based quadrature rules for computing integrals can be seen as a special case of random feature expansions for positive definite kernels, for a particular decomposition that always exists for such kernels. We provide a theoretical analysis of the number of required samples for a given approximation error, leading to both upper and lower bounds that are based solely on the eigenvalues of the associated integral operator and match up to logarithmic terms. In particular, we show that the upper bound may be obtained from independent and identically distributed samples from a specific non-uniform distribution, while the lower bound if valid for any set of points. Applying our results to kernel-based quadrature, while our results are fairly general, we recover known upper and lower bounds for the special cases of Sobolev spaces. Moreover, our results extend to the more general problem of full function approximations (beyond simply computing an integral), with results in $L_2$- and $L_\infty$-norm that match known results for special cases. Applying our results to random features, we show an improvement of the number of random features needed to preserve the generalization guarantees for learning with Lipshitz-continuous losses.

JMLR Other · 发表于 2017

On the Consistency of Ordinal Regression Methods

Fabian Pedregosa, Francis Bach, Alexandre Gramfort

摘要

Many of the ordinal regression models that have been proposed in the literature can be seen as methods that minimize a convex surrogate of the zero-one, absolute, or squared loss functions. A key property that allows to study the statistical implications of such approximations is that of Fisher consistency. Fisher consistency is a desirable property for surrogate loss functions and implies that in the population setting, i.e., if the probability distribution that generates the data were available, then optimization of the surrogate would yield the best possible model. In this paper we will characterize the Fisher consistency of a rich family of surrogate loss functions used in the context of ordinal regression, including support vector ordinal regression, ORBoosting and least absolute deviation. We will see that, for a family of surrogate loss functions that subsumes support vector ordinal regression and ORBoosting, consistency can be fully characterized by the derivative of a real-valued function at zero, as happens for convex margin-based surrogates in binary classification. We also derive excess risk bounds for a surrogate of the absolute error that generalize existing risk bounds for binary classification. Finally, our analysis suggests a novel surrogate of the squared error loss. We compare this novel surrogate with competing approaches on 9 different datasets. Our method shows to be highly competitive in practice, outperforming the least squares loss on 7 out of 9 datasets.

JMLR Other · 发表于 2017

On Perturbed Proximal Gradient Algorithms

Yves F. Atchadé, Gersende Fort, Eric Moulines

摘要

We study a version of the proximal gradient algorithm for which the gradient is intractable and is approximated by Monte Carlo methods (and in particular Markov Chain Monte Carlo). We derive conditions on the step size and the Monte Carlo batch size under which convergence is guaranteed: both increasing batch size and constant batch size are considered. We also derive non- asymptotic bounds for an averaged version. Our results cover both the cases of biased and unbiased Monte Carlo approximation. To support our findings, we discuss the inference of a sparse generalized linear model with random effect and the problem of learning the edge structure and parameters of sparse undirected graphical models.

Econometrica Other · 发表于 2017

On Monotone Recursive Preferences

Antoine Bommier, Asen Kochov, François Le Grand

摘要

We explore the set of preferences defined over temporal lotteries in an infinite horizon setting. We provide utility representations for all preferences that are both recursive and monotone. Our results indicate that the class of monotone recursive preferences includes Uzawa and risk-sensitive preferences, but leaves aside several of the recursive models suggested by Epstein and Zin (1989). Our representation result is derived in great generality using Lundberg (1982, 1985)'s work on functional equations. *(abstract via OpenAlex)*

JMLR Bayesian Methods · 发表于 2017

On Markov chain Monte Carlo methods for tall data

Rémi Bardenet, Arnaud Doucet, Chris Holmes

摘要

Markov chain Monte Carlo methods are often deemed too computationally intensive to be of any practical use for big data applications, and in particular for inference on datasets containing a large number $n$ of individual data points, also known as tall datasets. In scenarios where data are assumed independent, various approaches to scale up the Metropolis- Hastings algorithm in a Bayesian inference context have been recently proposed in machine learning and computational statistics. These approaches can be grouped into two categories: divide-and-conquer approaches and, subsampling-based algorithms. The aims of this article are as follows. First, we present a comprehensive review of the existing literature, commenting on the underlying assumptions and theoretical guarantees of each method. Second, by leveraging our understanding of these limitations, we propose an original subsampling-based approach relying on a control variate method which samples under regularity conditions from a distribution provably close to the posterior distribution of interest, yet can require less than $O(n)$ data point likelihood evaluations at each iteration for certain statistical models in favourable scenarios. Finally, we emphasize that we have only been able so far to propose subsampling-based methods which display good performance in scenarios where the Bernstein-von Mises approximation of the target posterior distribution is excellent. It remains an open challenge to develop such methods in scenarios where the Bernstein-von Mises approximation is poor.

JMLR Other · 发表于 2017

On Computationally Tractable Selection of Experiments in Measurement-Constrained Regression Models

Yining Wang, Adams Wei Yu, Aarti Singh

摘要

We derive computationally tractable methods to select a small subset of experiment settings from a large pool of given design points. The primary focus is on linear regression models, while the technique extends to generalized linear models and Delta's method (estimating functions of linear regression models) as well. The algorithms are based on a continuous relaxation of an otherwise intractable combinatorial optimization problem, with sampling or greedy procedures as post-processing steps. Formal approximation guarantees are established for both algorithms, and numerical results on both synthetic and real-world data confirm the effectiveness of the proposed methods.

Biometrika Bayesian Methods Functional Data · 发表于 2017

OUP accepted manuscript

not listed

摘要

SummaryBayes classifiers for functional data pose a challenge. One difficulty is that probability density functions do not exist for functional data, so the classical Bayes classifier using density quotients needs to be modified. We propose to use density ratios of projections onto a sequence of eigenfunctions that are common to the groups to be classified. The density ratios are then factorized into density ratios of individual projection scores, reducing the classification problem to obtaining a series of one-dimensional nonparametric density estimates. The proposed classifiers can be viewed as an extension to functional data of some of the earliest nonparametric Bayes classifiers that were based on simple density ratios in the one-dimensional case. By means of the factorization of the density quotients, the curse of dimensionality that would otherwise severely affect Bayes classifiers for functional data can be avoided. We demonstrate that in the case of Gaussian functional data, the proposed functional Bayes classifier reduces to a functional version of the classical quadratic discriminant. A study of the asymptotic behaviour of the proposed classifiers in the large-sample limit shows that under certain conditions the misclassification rate converges to zero, a phenomenon that has been referred to as perfect classification. The proposed classifiers also perform favourably in finite-sample settings, as we demonstrate through comparisons with other functional classifiers in simulations and various data applications, including spectral data, functional magnetic resonance imaging data from attention deficit hyperactivity disorder patients, and yeast gene expression data. *(abstract via OpenAlex)*

Econometrica Nonparametric & Semiparametric · 发表于 2017

Nonparametric Stochastic Discount Factor Decomposition

Timothy M. Christensen

摘要

Stochastic discount factor (SDF) processes in dynamic economies admit a permanent-transitory decomposition in which the permanent component characterizes pricing over long investment horizons. This paper introduces an empirical framework to analyze the permanent-transitory decomposition of SDF processes. Specifically, we show how to estimate nonparametrically the solution to the Perron-Frobenius eigenfunction problem of Hansen and Scheinkman (2009). Our empirical framework allows researchers to (i) recover the time series of the estimated permanent and transitory components and (ii) estimate the yield and the change of measure which characterize pricing over long investment horizons. We also introduce nonparametric estimators of the continuation value function in a class of models with recursive preferences by reinterpreting the value function recursion as a nonlinear Perron-Frobenius problem. We establish consistency and convergence rates of the eigenfunction estimators and asymptotic normality of the eigenvalue estimator and estimators of related functionals. As an application, we study an economy where the representative agent is endowed with recursive preferences, allowing for general (nonlinear) consumption and earnings growth dynamics. *(abstract via OpenAlex)*

JMLR Spatial & Time Series Nonparametric & Semiparametric Econometrics & Economics · 发表于 2017

Nonparametric Risk Bounds for Time-Series Forecasting

Daniel J. McDonald, Cosma Rohilla Shalizi, Mark Schervish

摘要

We derive generalization error bounds for traditional time- series forecasting models. Our results hold for many standard forecasting tools including autoregressive models, moving average models, and, more generally, linear state-space models. These non-asymptotic bounds need only weak assumptions on the data-generating process, yet allow forecasters to select among competing models and to guarantee, with high probability, that their chosen model will perform well. We motivate our techniques with and apply them to standard economic and financial forecasting tools---a GARCH model for predicting equity volatility and a dynamic stochastic general equilibrium model (DSGE), the standard tool in macroeconomic forecasting. We demonstrate in particular how our techniques can aid forecasters and policy makers in choosing models which behave well under uncertainty and mis-specification.

Econometrica Causal Inference Nonparametric & Semiparametric · 发表于 2017

Nonparametric Instrumental Variable Estimation Under Monotonicity

Denis Chetverikov, Daniel Wilhelm

摘要

The ill-posedness of the inverse problem of recovering a regression function in a nonparametric instrumental variable model leads to estimators that may suffer from a very slow, logarithmic rate of convergence. In this paper, we show that restricting the problem to models with monotone regression functions and monotone instruments significantly weakens the ill-posedness of the problem. In stark contrast to the existing literature, the presence of a monotone instrument implies boundedness of our measure of ill-posedness when restricted to the space of monotone functions. Based on this result we derive a novel non-asymptotic error bound for the constrained estimator that imposes monotonicity of the regression function. For a given sample size, the bound is independent of the degree of ill-posedness as long as the regression function is not too steep. As an implication, the bound allows us to show that the constrained estimator converges at a fast, polynomial rate, independently of the degree of ill-posedness, in a large, but slowly shrinking neighborhood of constant functions. Our simulation study demonstrates significant finite-sample performance gains from imposing monotonicity even when the regression function is rather far from being a constant. We apply the constrained estimator to the problem of estimating gasoline demand functions from U.S. data. *(abstract via OpenAlex)*

JMLR Other · 发表于 2017

Non-parametric Policy Search with Limited Information Loss

Herke van Hoof, Gerhard Neumann, Jan Peters

摘要

Learning complex control policies from non-linear and redundant sensory input is an important challenge for reinforcement learning algorithms. Non-parametric methods that approximate values functions or transition models can address this problem, by adapting to the complexity of the data set. Yet, many current non-parametric approaches rely on unstable greedy maximization of approximate value functions, which might lead to poor convergence or oscillations in the policy update. A more robust policy update can be obtained by limiting the information loss between successive state-action distributions. In this paper, we develop a policy search algorithm with policy updates that are both robust and non-parametric. Our method can learn non- parametric control policies for infinite horizon continuous Markov decision processes with non-linear and redundant sensory representations. We investigate how we can use approximations of the kernel function to reduce the time requirements of the demanding non-parametric computations. In our experiments, we show the strong performance of the proposed method, and how it can be approximated efficiently. Finally, we show that our algorithm can learn a real-robot under-powered swing-up task directly from image data.

Econometrica Econometrics & Economics Networks & Graphs · 发表于 2017

Networks in Conflict: Theory and Evidence From the Great War of Africa

Michael D. König, Dominic Rohner, Mathias Thoenig, Fabrizio Zilibotti

摘要

We study from both a theoretical and an empirical perspective how a network of military alliances and enmities affects the intensity of a conflict. The model combines elements from network theory and from the politico-economic theory of conflict. We obtain a closed-form characterization of the Nash equilibrium. Using the equilibrium conditions, we perform an empirical analysis using data on the Second Congo War, a conflict that involves many groups in a complex network of informal alliances and rivalries. The estimates of the fighting externalities are then used to infer the extent to which the conflict intensity can be reduced through (i) dismantling specific fighting groups involved in the conflict; (ii) weapon embargoes; (iii) interventions aimed at pacifying animosity among groups. Finally, with the aid of a random utility model, we study how policy shocks can induce a reshaping of the network structure. *(abstract via OpenAlex)*

JMLR Other · 发表于 2017

Nearly optimal classification for semimetrics

Lee-Ad Gottlieb, Aryeh Kontorovich, Pinhas Nisnevitch

摘要

We initiate the rigorous study of classification in semimetric spaces, which are point sets with a distance function that is non-negative and symmetric, but need not satisfy the triangle inequality. We define the density dimension dens and discover that it plays a central role in the statistical and algorithmic feasibility of learning in semimetric spaces. We compute this quantity for several widely used semimetrics and present nearly optimal sample compression algorithms, which are then used to obtain generalization guarantees, including fast rates. Our claim of near-optimality holds in both computational and statistical senses. When the sample has radius $R$ and margin $\gamma$, we show that it can be compressed down to roughly $d=(R/\gamma)^{\text{dens}}$ points, and further that finding a significantly better compression is algorithmically intractable unless P=NP. This compression implies generalization via standard Occam-type arguments, to which we provide a nearly matching lower bound.

JMLR Optimization & Learning Theory · 发表于 2017

Multiscale Strategies for Computing Optimal Transport

Samuel Gerber, Mauro Maggioni

摘要

This paper presents a multiscale approach to efficiently compute approximate optimal transport plans between point sets. It is particularly well-suited for point sets that are in high- dimensions, but are close to being intrinsically low- dimensional. The approach is based on an adaptive multiscale decomposition of the point sets. The multiscale decomposition yields a sequence of optimal transport problems, that are solved in a top-to-bottom fashion from the coarsest to the finest scale. We provide numerical evidence that this multiscale approach scales approximately linearly, in time and memory, in the number of nodes, instead of quadratically or worse for a direct solution. Empirically, the multiscale approach results in less than one percent relative error in the objective function. Furthermore, the multiscale plans constructed are of interest by themselves as they may be used to introduce novel features and notions of distances between point sets. An analysis of sets of brain MRI based on optimal transport distances illustrates the effectiveness of the proposed method on a real world data set. The application demonstrates that multiscale optimal transport distances have the potential to improve on state-of-the-art metrics currently used in computational anatomy.

Econometrica Econometrics & Economics · 发表于 2017

Money as a Unit of Account

Matthias Doepke, Martin Schneider

摘要

We develop a theory that rationalizes the use of a dominant unit of account in an economy.Agents enter into non-contingent contracts with a variety of business partners.Trade unfolds sequentially in credit chains and is subject to random matching.By using a dominant unit of account, agents can lower their exposure to relative price risk, avoid costly default, and create more total surplus.We discuss conditions under which it is optimal to adopt circulating government paper as the dominant unit of account, and the optimal choice of "currency areas" when there is variation in the intensity of trade within and across regions. *(abstract via OpenAlex)*

JMLR Optimization & Learning Theory · 发表于 2017

Minimax Filter: Learning to Preserve Privacy from Inference Attacks

Jihun Hamm

摘要

Preserving privacy of continuous and/or high-dimensional data such as images, videos and audios, can be challenging with syntactic anonymization methods which are designed for discrete attributes. Differentially privacy, which uses a more rigorous definition of privacy loss, has shown more success in sanitizing continuous data. However, both syntactic and differential privacy are susceptible to inference attacks, i.e., an adversary can accurately infer sensitive attributes from sanitized data. The paper proposes a novel filter-based mechanism which preserves privacy of continuous and high-dimensional attributes against inference attacks. Finding the optimal utility-privacy tradeoff is formulated as a min-diff-max optimization problem. The paper provides an ERM-like analysis of the generalization error and also a practical algorithm to perform minimax optimization. In addition, the paper proposes a noisy minimax filter which combines minimax filter and differentially-private mechanism. Advantages of the method over purely noisy mechanisms is explained and demonstrated with examples. Experiments with several real-world tasks including facial expression classification, speech emotion classification, and activity classification from motion, show that the minimax filter can simultaneously achieve similar or higher target task accuracy and lower inference accuracy, often significantly lower than previous methods.

JMLR Nonparametric & Semiparametric · 发表于 2017

Minimax Estimation of Kernel Mean Embeddings

Ilya Tolstikhin, Bharath K. Sriperumbudur, Krikamol Muandet

摘要

In this paper, we study the minimax estimation of the Bochner integral \[ \mu_k(P) := \int_\mathcal{X} k(\cdot,x)\, dP(x), \] also called the kernel mean embedding, based on random samples drawn i.i.d. from $P$, where $k:\mathcal{X}\times\mathcal{X}\rightarrow \mathbb{R}$ is a positive definite kernel. Various estimators (including the empirical estimator), $\hat{\theta}_n$ of $\mu_k(P)$ are studied in the literature wherein all of them satisfy $\|\hat{\theta}_n-\mu_k(P)\|_{\mathcal{H}_k}=O_P(n^{-1/2})$ with $\mathcal{H}_k$ being the reproducing kernel Hilbert space induced by $k$. The main contribution of the paper is in showing that the above mentioned rate of $n^{-1/2}$ is minimax in $\|\cdot\|_{\mathcal{H}_k}$ and $\|\cdot\|_{L^2(\mathbb{R}^d)}$-norms over the class of discrete measures and the class of measures that has an infinitely differentiable density, with $k$ being a continuous translation- invariant kernel on $\mathbb{R}^d$. The interesting aspect of this result is that the minimax rate is independent of the smoothness of the kernel and the density of $P$ (if it exists).

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 页