2016 年的期刊文章 · 第 34 页

共 1345 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2016 到 2016。

按期刊、按主题看 →

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 页

JMLR Latent Variables & Mixtures · 发表于 2016

Adjusting for Chance Clustering Comparison Measures

Simone Romano, Nguyen Xuan Vinh, James Bailey, Karin Verspoor

摘要

Adjusted for chance measures are widely used to compare partitions/clusterings of the same data set. In particular, the Adjusted Rand Index (ARI) based on pair-counting, and the Adjusted Mutual Information (AMI) based on Shannon information theory are very popular in the clustering community. Nonetheless it is an open problem as to what are the best application scenarios for each measure and guidelines in the literature for their usage are sparse, with the result that users often resort to using both. Generalized Information Theoretic (IT) measures based on the Tsallis entropy have been shown to link pair- counting and Shannon IT measures. In this paper, we aim to bridge the gap between adjustment of measures based on pair- counting and measures based on information theory. We solve the key technical challenge of analytically computing the expected value and variance of generalized IT measures. This allows us to propose adjustments of generalized IT measures, which reduce to well known adjusted clustering comparison measures as special cases. Using the theory of generalized IT measures, we are able to propose the following guidelines for using ARI and AMI as external validation indices: ARI should be used when the reference clustering has large equal sized clusters; AMI should be used when the reference clustering is unbalanced and there exist small clusters.

JMLR Optimization & Learning Theory · 发表于 2016

Addressing Environment Non-Stationarity by Repeating Q-learning Updates

Sherief Abdallah, Michael Kaisers

摘要

Q-learning (QL) is a popular reinforcement learning algorithm that is guaranteed to converge to optimal policies in Markov decision processes. However, QL exhibits an artifact: in expectation, the effective rate of updating the value of an action depends on the probability of choosing that action. In other words, there is a tight coupling between the learning dynamics and underlying execution policy. This coupling can cause performance degradation in noisy non-stationary environments.

JMLR High-Dimensional · 发表于 2016

Adaptive Lasso and group-Lasso for functional Poisson regression

Stéphane Ivanoff, Franck Picard, Vincent Rivoirard

摘要

High dimensional Poisson regression has become a standard framework for the analysis of massive counts datasets. In this work we estimate the intensity function of the Poisson regression model by using a dictionary approach, which generalizes the classical basis approach, combined with a Lasso or a group-Lasso procedure. Selection depends on penalty weights that need to be calibrated. Standard methodologies developed in the Gaussian framework can not be directly applied to Poisson models due to heteroscedasticity. Here we provide data-driven weights for the Lasso and the group-Lasso derived from concentration inequalities adapted to the Poisson case. We show that the associated Lasso and group-Lasso procedures satisfy fast and slow oracle inequalities. Simulations are used to assess the empirical performance of our procedure, and an original application to the analysis of Next Generation Sequencing data is provided.

JMLR High-Dimensional · 发表于 2016

A Well-Conditioned and Sparse Estimation of Covariance and Inverse Covariance Matrices Using a Joint Penalty

Ashwini Maurya

摘要

We develop a method for estimating well-conditioned and sparse covariance and inverse covariance matrices from a sample of vectors drawn from a sub-Gaussian distribution in high dimensional setting. The proposed estimators are obtained by minimizing the quadratic loss function and joint penalty of $\ell_1$ norm and variance of its eigenvalues. In contrast to some of the existing methods of covariance and inverse covariance matrix estimation, where often the interest is to estimate a sparse matrix, the proposed method is flexible in estimating both a sparse and well-conditioned covariance matrix simultaneously. The proposed estimators are optimal in the sense that they achieve the mini-max rate of estimation in operator norm for the underlying class of covariance and inverse covariance matrices. We give a very fast algorithm for computation of these covariance and inverse covariance matrices which is easily scalable to large scale data analysis problems. The simulation study for varying sample sizes and variables shows that the proposed estimators performs better than several other estimators for various choices of structured covariance and inverse covariance matrices. We also use our proposed estimator for tumor tissues classification using gene expression data and compare its performance with some other classification methods.

JMLR Other · 发表于 2016

A Variational Approach to Path Estimation and Parameter Inference of Hidden Diffusion Processes

Tobias Sutter, Arnab Ganguly, Heinz Koeppl

摘要

We consider a hidden Markov model, where the signal process, given by a diffusion, is only indirectly observed through some noisy measurements. The article develops a variational method for approximating the hidden states of the signal process given the full set of observations. This, in particular, leads to systematic approximations of the smoothing densities of the signal process. The paper then demonstrates how an efficient inference scheme, based on this variational approach to the approximation of the hidden states, can be designed to estimate the unknown parameters of stochastic differential equations. Two examples at the end illustrate the efficacy and the accuracy of the presented method.

JMLR Nonparametric & Semiparametric · 发表于 2016

A Unifying Framework in Vector-valued Reproducing Kernel Hilbert Spaces for Manifold Regularization and Co-Regularized Multi-view Learning

Hà Quang Minh, Loris Bazzani, Vittorio Murino

摘要

This paper presents a general vector-valued reproducing kernel Hilbert spaces (RKHS) framework for the problem of learning an unknown functional dependency between a structured input space and a structured output space. Our formulation encompasses both Vector-valued Manifold Regularization and Co-regularized Multi- view Learning, providing in particular a unifying framework linking these two important learning approaches. In the case of the least square loss function, we provide a closed form solution, which is obtained by solving a system of linear equations. In the case of Support Vector Machine (SVM) classification, our formulation generalizes in particular both the binary Laplacian SVM to the multi-class, multi-view settings and the multi-class Simplex Cone SVM to the semi-supervised, multi-view settings. The solution is obtained by solving a single quadratic optimization problem, as in standard SVM, via the Sequential Minimal Optimization (SMO) approach. Empirical results obtained on the task of object recognition, using several challenging data sets, demonstrate the competitiveness of our algorithms compared with other state-of-the-art methods.

JMLR Other · 发表于 2016

A Unified View on Multi-class Support Vector Classification

Ürün Doğan, Tobias Glasmachers, Christian Igel

摘要

A unified view on multi-class support vector machines (SVMs) is presented, covering most prominent variants including the one- vs-all approach and the algorithms proposed by Weston & Watkins, Crammer & Singer, Lee, Lin, & Wahba, and Liu & Yuan. The unification leads to a template for the quadratic training problems and new multi-class SVM formulations. Within our framework, we provide a comparative analysis of the various notions of multi-class margin and margin-based loss. In particular, we demonstrate limitations of the loss function considered, for instance, in the Crammer & Singer machine.

Econometrica Econometrics & Economics · 发表于 2016

A Theory of Macroprudential Policies in the Presence of Nominal Rigidities

Emmanuel Farhi, Iván Werning

摘要

We propose a theory of monetary policy and macroprudential interventions in financial markets. We focus on economies with nominal rigidities in goods and labor markets and subject to constraints on monetary policy, such as the zero lower bound or fixed exchange rates. We identify an aggregate demand externality that can be corrected by macroprudential interventions in financial markets. Ex post, the distribution of wealth across agents affects aggregate demand and output. Ex ante, however, these effects are not internalized in private financial decisions. We provide a simple formula for the required financial interventions that depends on a small number of measurable sufficient statistics. We also characterize optimal monetary policy. We extend our framework to incorporate pecuniary externalities, providing a unified approach to both externalities. Finally, we provide a number of applications which illustrate the relevance of our theory. *(abstract via OpenAlex)*

JMLR Other · 发表于 2016

A Statistical Perspective on Randomized Sketching for Ordinary Least-Squares

Garvesh Raskutti, Michael W. Mahoney

摘要

We consider statistical as well as algorithmic aspects of solving large-scale least-squares (LS) problems using randomized sketching algorithms. For a LS problem with input data $(X, Y) \in \mathbb{R}^{n \times p} \times \mathbb{R}^n$, sketching algorithms use a sketching matrix, $S\in\mathbb{R}^{r \times n}$, where $r \ll n$. Then, rather than solving the LS problem using the full data $(X,Y)$, sketching algorithms solve the LS problem using only the sketched data $(SX, SY)$. Prior work has typically adopted an algorithmic perspective, in that it has made no statistical assumptions on the input $X$ and $Y$, and instead it has been assumed that the data $(X,Y)$ are fixed and worst-case (WC). Prior results show that, when using sketching matrices such as random projections and leverage-score sampling algorithms, with $p \lesssim r \ll n$, the WC error is the same as solving the original problem, up to a small constant. From a statistical perspective, we typically consider the mean-squared error performance of randomized sketching algorithms, when data $(X, Y)$ are generated according to a statistical linear model $Y = X \beta + \epsilon$, where $\epsilon$ is a noise process. In this paper, we provide a rigorous comparison of both perspectives leading to insights on how they differ. To do this, we first develop a framework for assessing, in a unified manner, algorithmic and statistical aspects of randomized sketching methods. We then consider the statistical prediction efficiency (PE) and the statistical residual efficiency (RE) of the sketched LS estimator; and we use our framework to provide upper bounds for several types of random projection and random sampling sketching algorithms. Among other results, we show that the RE can be upper bounded when $p \lesssim r \ll n$ while the PE typically requires the sample size $r$ to be substantially larger. Lower bounds developed in subsequent results show that our upper bounds on PE can not be improved. (A preliminary version of this paper appeared as Raskutti and Mahoney (2014, 2015).)

Econometrica Other · 发表于 2016

A Rational Theory of Mutual Funds' Attention Allocation

Marcin Kacperczyk, Stijn Van Nieuwerburgh, Laura Veldkamp

摘要

The question of whether and how mutual fund managers provide valuable services for their clients motivates one of the largest literatures in finance. One candidate explanation is that funds process information about future asset values and use that information to invest in high-valued assets. But formal theories are scarce because information choice models with many assets are difficult to solve as well as difficult to test. This paper tackles both problems by developing a new attention allocation model that uses the state of the business cycle to predict information choices, which in turn, predict observable patterns of portfolio investments and returns. The predictions about fund portfolios ’ covariance with payoff shocks, cross-fund portfolio and return dispersion, and their excess returns are all supported by the data. In turn, these findings offer new evidence that some investment managers have skill and that attention is allocated rationally. *(abstract via OpenAlex)*

JMLR High-Dimensional · 发表于 2016

A Practical Scheme and Fast Algorithm to Tune the Lasso With Optimality Guarantees

Michael Chichignoud, Johannes Lederer, Martin J. Wainwright

摘要

We introduce a novel scheme for choosing the regularization parameter in high-dimensional linear regression with Lasso. This scheme, inspired by Lepski’s method for bandwidth selection in non-parametric regression, is equipped with both optimal finite-sample guarantees and a fast algorithm. In particular, for any design matrix such that the Lasso has low sup-norm error under an “oracle choice” of the regularization parameter, we show that our method matches the oracle performance up to a small constant factor, and show that it can be implemented by performing simple tests along a single Lasso path. By applying the Lasso to simulated and real data, we find that our novel scheme can be faster and more accurate than standard schemes such as Cross-Validation.

JMLR High-Dimensional Optimization & Learning Theory · 发表于 2016

A Note on the Sample Complexity of the Er-SpUD Algorithm by Spielman, Wang and Wright for Exact Recovery of Sparsely Used Dictionaries

Radoslaw Adamczak

摘要

We consider the problem of recovering an invertible $n \times n$ matrix $A$ and a sparse $n \times p$ random matrix $X$ based on the observation of $Y = AX$ (up to a scaling and permutation of columns of $A$ and rows of $X$). Using only elementary tools from the theory of empirical processes we show that a version of the Er-SpUD algorithm by Spielman, Wang and Wright with high probability recovers $A$ and $X$ exactly, provided that $p \ge Cn\log n$, which is optimal up to the constant $C$.

Econometrica Other · 发表于 2016

A Note on Comparative Ambiguity Aversion and Justifiability

P. Battigalli, S. Cerreia-Vioglio, F. Maccheroni, M. Marinacci

摘要

We consider a decision maker who ranks actions according to the smooth ambiguity criterion of Klibanos et al. (2005). An action is justifiable if it is a best reply to some belief over probabilistic models. We show that higher ambiguity aversion expands the set of justifiable actions. In turn, this implies that higher ambiguity aversion expands the set of rationalizable actions of a game. Our results follow from a generalization of the duality lemma of Wald (1949) and Pearce (1984). *(abstract via OpenAlex)*

JMLR High-Dimensional Latent Variables & Mixtures · 发表于 2016

A New Algorithm and Theory for Penalized Regression-based Clustering

Chong Wu, Sunghoon Kwon, Xiaotong Shen, Wei Pan

摘要

Clustering is unsupervised and exploratory in nature. Yet, it can be performed through penalized regression with grouping pursuit, as demonstrated in Pan et al. (2013). In this paper, we develop a more efficient algorithm for scalable computation and a new theory of clustering consistency for the method. This algorithm, called DC-ADMM, combines difference of convex (DC) programming with the alternating direction method of multipliers (ADMM). This algorithm is shown to be more computationally efficient than the quadratic penalty based algorithm of Pan et al. (2013) because of the former's closed-form updating formulas. Numerically, we compare the DC- ADMM algorithm with the quadratic penalty algorithm to demonstrate its utility and scalability. Theoretically, we establish a finite-sample mis- clustering error bound for penalized regression based clustering with the $L_0$ constrained regularization in a general setting. On this ground, we provide conditions for clustering consistency of the penalized clustering method. As an end product, we put R package prclust implementing PRclust with various loss and grouping penalty functions available on GitHub and CRAN.

JMLR Networks & Graphs · 发表于 2016

A Network That Learns Strassen Multiplication

Veit Elser

摘要

We study neural networks whose only non-linear components are multipliers, to test a new training rule in a context where the precise representation of data is paramount. These networks are challenged to discover the rules of matrix multiplication, given many examples. By limiting the number of multipliers, the network is forced to discover the Strassen multiplication rules. This is the mathematical equivalent of finding low rank decompositions of the $n\times n$ matrix multiplication tensor, $M_n$. We train these networks with the conservative learning rule, which makes minimal changes to the weights so as to give the correct output for each input at the time the input-output pair is received. Conservative learning needs a few thousand examples to find the rank 7 decomposition of $M_2$, and $10^5$ for the rank 23 decomposition of $M_3$ (the lowest known). High precision is critical, especially for $M_3$, to discriminate between true decompositions and “border approximations".

JMLR Bayesian Methods Networks & Graphs · 发表于 2016

A Gibbs Sampler for Learning DAGs

Robert J. B. Goudie, Sach Mukherjee

摘要

We propose a Gibbs sampler for structure learning in directed acyclic graph (DAG) models. The standard Markov chain Monte Carlo algorithms used for learning DAGs are random-walk Metropolis-Hastings samplers. These samplers are guaranteed to converge asymptotically but often mix slowly when exploring the large graph spaces that arise in structure learning. In each step, the sampler we propose draws entire sets of parents for multiple nodes from the appropriate conditional distribution. This provides an efficient way to make large moves in graph space, permitting faster mixing whilst retaining asymptotic guarantees of convergence. The conditional distribution is related to variable selection with candidate parents playing the role of covariates or inputs. We empirically examine the performance of the sampler using several simulated and real data examples. The proposed method gives robust results in diverse settings, outperforming several existing Bayesian and frequentist methods. In addition, our empirical results shed some light on the relative merits of Bayesian and constraint- based methods for structure learning.

Econometrica Econometrics & Economics · 发表于 2016

A Geometric Approach to Nonlinear Econometric Models

Isaiah Andrews, Anna Mikusheva

摘要

Conventional tests for composite hypotheses in minimum distance models can be unreliable when the relationship between the structural and reduced‐form parameters is highly nonlinear. Such nonlinearity may arise for a variety of reasons, including weak identification. In this note, we begin by studying the problem of testing a “curved null” in a finite‐sample Gaussian model. Using the curvature of the model, we develop new finite‐sample bounds on the distribution of minimum‐distance statistics. These bounds allow us to construct tests for composite hypotheses which are uniformly asymptotically valid over a large class of data generating processes and structural models. *(abstract via OpenAlex)*

JMLR Optimization & Learning Theory Bayesian Methods · 发表于 2016

A General Framework for Constrained Bayesian Optimization using Information-based Search

José Miguel Hernández-Lobato, Michael A. Gelbart, Ryan P. Adams, Matthew W. Hoffman, Zoubin Ghahramani

摘要

We present an information-theoretic framework for solving global black-box optimization problems that also have black-box constraints. Of particular interest to us is to efficiently solve problems with decoupled constraints, in which subsets of the objective and constraint functions may be evaluated independently. For example, when the objective is evaluated on a CPU and the constraints are evaluated independently on a GPU. These problems require an acquisition function that can be separated into the contributions of the individual function evaluations. We develop one such acquisition function and call it Predictive Entropy Search with Constraints (PESC). PESC is an approximation to the expected information gain criterion and it compares favorably to alternative approaches based on improvement in several synthetic and real- world problems. In addition to this, we consider problems with a mix of functions that are fast and slow to evaluate. These problems require balancing the amount of time spent in the meta- computation of PESC and in the actual evaluation of the target objective. We take a bounded rationality approach and develop a partial update for PESC which trades off accuracy against speed. We then propose a method for adaptively switching between the partial and full updates for PESC. This allows us to interpolate between versions of PESC that are efficient in terms of function evaluations and those that are efficient in terms of wall-clock time. Overall, we demonstrate that PESC is an effective algorithm that provides a promising direction towards a unified solution for constrained Bayesian optimization.

JMLR Other · 发表于 2016

A General Framework for Consistency of Principal Component Analysis

Dan Shen, Haipeng Shen, J. S. Marron

摘要

A general asymptotic framework is developed for studying consistency properties of principal component analysis (PCA). Our framework includes several previously studied domains of asymptotics as special cases and allows one to investigate interesting connections and transitions among the various domains. More importantly, it enables us to investigate asymptotic scenarios that have not been considered before, and gain new insights into the consistency, subspace consistency and strong inconsistency regions of PCA and the boundaries among them. We also establish the corresponding convergence rate within each region. Under general spike covariance models, the dimension (or number of variables) discourages the consistency of PCA, while the sample size and spike information (the relative size of the population eigenvalues) encourage PCA consistency. Our framework nicely illustrates the relationship among these three types of information in terms of dimension, sample size and spike size, and rigorously characterizes how their relationships affect PCA consistency.

Econometrica Other · 发表于 2016

A Dynamic Model of Demand for Houses and Neighborhoods

Patrick Bayer, Robert McMillan, Alvin Murphy, Christopher Timmins

摘要

This paper develops a dynamic model of neighborhood choice along with a computationally light multi-step estimator.The proposed empirical framework captures observed and unobserved preference heterogeneity across households and locations in a flexible way.The model is estimated using a newly assembled data set that matches demographic information from mortgage applications to the universe of housing transactions in the San Francisco Bay Area from 1994-2004.The results provide the first estimates of the marginal willingness to pay for several non-marketed amenities -neighborhood air pollution, violent crime and racial composition -in a dynamic framework.Comparing these estimates with those from a static version of the model highlights several important biases that arise when dynamic considerations are ignored. *(abstract via OpenAlex)*

JMLR Other · 发表于 2016

A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights

Weijie Su, Stephen Boyd, Emmanuel J. Candès

摘要

We derive a second-order ordinary differential equation (ODE) which is the limit of Nesterov's accelerated gradient method. This ODE exhibits approximate equivalence to Nesterov's scheme and thus can serve as a tool for analysis. We show that the continuous time ODE allows for a better understanding of Nesterov's scheme. As a byproduct, we obtain a family of schemes with similar convergence rates. The ODE interpretation also suggests restarting Nesterov's scheme leading to an algorithm, which can be rigorously proven to converge at a linear rate whenever the objective is strongly convex.

JMLR Other · 发表于 2016

A Consistent Information Criterion for Support Vector Machines in Diverging Model Spaces

Xiang Zhang, Yichao Wu, Lan Wang, Runze Li

摘要

Information criteria have been popularly used in model selection and proved to possess nice theoretical properties. For classification, Claeskens et al. (2880) proposed support vector machine information criterion for feature selection and provided encouraging numerical evidence. Yet no theoretical justification was given there. This work aims to fill the gap and to provide some theoretical justifications for support vector machine information criterion in both fixed and diverging model spaces. We first derive a uniform convergence rate for the support vector machine solution and then show that a modification of the support vector machine information criterion achieves model selection consistency even when the number of features diverges at an exponential rate of the sample size. This consistency result can be further applied to selecting the optimal tuning parameter for various penalized support vector machine methods. Finite-sample performance of the proposed information criterion is investigated using Monte Carlo studies and one real-world gene selection problem.

JMLR Optimization & Learning Theory · 发表于 2016

A Closer Look at Adaptive Regret

Dmitry Adamskiy, Wouter M. Koolen, Alexey Chernov, Vladimir Vovk

摘要

For the prediction with expert advice setting, we consider methods to construct algorithms that have low adaptive regret. The adaptive regret of an algorithm on a time interval $[t_1,t_2]$ is the loss of the algorithm minus the loss of the best expert over that interval. Adaptive regret measures how well the algorithm approximates the best expert locally, and so is different from, although closely related to, both the classical regret, measured over an initial time interval $[1,t]$, and the tracking regret, where the algorithm is compared to a good sequence of experts over $[1,t]$. We investigate two existing intuitive methods for deriving algorithms with low adaptive regret, one based on specialist experts and the other based on restarts. Quite surprisingly, we show that both methods lead to the same algorithm, namely Fixed Share, which is known for its tracking regret. We provide a thorough analysis of the adaptive regret of Fixed Share. We obtain the exact worst-case adaptive regret for Fixed Share, from which the classical tracking bounds follow. We prove that Fixed Share is optimal for adaptive regret: the worst-case adaptive regret of any algorithm is at least that of an instance of Fixed Share.

JMLR Latent Variables & Mixtures · 发表于 2016

A Characterization of Linkage-Based Hierarchical Clustering

Margareta Ackerman, Shai Ben-David

摘要

The class of linkage-based algorithms is perhaps the most popular class of hierarchical algorithms. We identify two properties of hierarchical algorithms, and prove that linkage- based algorithms are the only ones that satisfy both of these properties. Our characterization clearly delineates the difference between linkage-based algorithms and other hierarchical methods. We formulate an intuitive notion of locality of a hierarchical algorithm that distinguishes between linkage-based and global hierarchical algorithms like bisecting $k$-means, and prove that popular divisive hierarchical algorithms produce clusterings that cannot be produced by any linkage-based algorithm.

JMLR Bayesian Methods · 发表于 2016

A Bounded p-norm Approximation of Max-Convolution for Sub-Quadratic Bayesian Inference on Additive Factors

Julianus Pfeuffer, Oliver Serang

摘要

Max-convolution is an important problem closely resembling standard convolution; as such, max-convolution occurs frequently across many fields. Here we extend the method with fastest known worst-case runtime, which can be applied to nonnegative vectors by numerically approximating the Chebyshev norm $\| \cdot \|_\infty$, and use this approach to derive two numerically stable methods based on the idea of computing $p$-norms via fast convolution: The first method proposed, with runtime in $O( k \log(k) \log(\log(k)) )$ (which is less than $18 k \log(k)$ for any vectors that can be practically realized), uses the $p$-norm as a direct approximation of the Chebyshev norm. The second approach proposed, with runtime in $O( k \log(k) )$ (although in practice both perform similarly), uses a novel null space projection method, which extracts information from a sequence of $p$-norms to estimate the maximum value in the vector (this is equivalent to querying a small number of moments from a distribution of bounded support in order to estimate the maximum). The $p$-norm approaches are compared to one another and are shown to compute an approximation of the Viterbi path in a hidden Markov model where the transition matrix is a Toeplitz matrix; the runtime of approximating the Viterbi path is thus reduced from $O( n k^2 )$ steps to $O( n k \log(k))$ steps in practice, and is demonstrated by inferring the U.S. unemployment rate from the S&P 500 stock index.

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 页