2016 年的期刊文章 · 第 31 页

共 1345 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2016 到 2016。

按期刊、按主题看 →

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 页

JMLR Nonparametric & Semiparametric Optimization & Learning Theory · 发表于 2016

Large Scale Online Kernel Learning

Jing Lu, Steven C.H. Hoi, Jialei Wang, Peilin Zhao, Zhi-Yong Liu

摘要

In this paper, we present a new framework for large scale online kernel learning, making kernel methods efficient and scalable for large-scale online learning applications. Unlike the regular budget online kernel learning scheme that usually uses some budget maintenance strategies to bound the number of support vectors, our framework explores a completely different approach of kernel functional approximation techniques to make the subsequent online learning task efficient and scalable. Specifically, we present two different online kernel machine learning algorithms: (i) Fourier Online Gradient Descent (FOGD) algorithm that applies the random Fourier features for approximating kernel functions; and (ii) Nyström Online Gradient Descent (NOGD) algorithm that applies the Nyström method to approximate large kernel matrices. We explore these two approaches to tackle three online learning tasks: binary classification, multi-class classification, and regression. The encouraging results of our experiments on large-scale datasets validate the effectiveness and efficiency of the proposed algorithms, making them potentially more practical than the family of existing budget online kernel learning approaches.

Econometrica Econometrics & Economics · 发表于 2016

Large Market Asymptotics for Differentiated Product Demand Estimators With Economic Models of Supply

Timothy B. Armstrong

摘要

IO economists often estimate demand for differentiated products using data sets with a small number of large markets. By modeling demand as depending on a small number of product characteristics, one might hope to obtain increasingly precise estimates of demand parameters as the number of products in a single market grows large. In this paper, I address the question of consistency and asymptotic distributions of IV estimates of demand in a small number of markets as the number of products increases in some commonly used demand models under conditions on economic primitives. I show that, under the common assumption of a Bertrand-Nash equilibrium in prices, product characteristics lose their identifying power as price instruments in the limit in many of these models, giving inconsistent estimates in these cases. I find that consistent estimates can still be obtained for many of the cases I consider, but care must be taken in modeling demand and choosing instruments. For cases where consistent estimates can be obtained, I provide sufficient conditions for consistency and asymptotic normality of estimates of parameters and counterfactual outcomes. A monte carlo study confirms that the asymptotic results provide an accurate description *(abstract via OpenAlex)*

Econometrica Econometrics & Economics · 发表于 2016

Large Contests

Wojciech Olszewski, Ron Siegel

摘要

We consider contests with many, possibly heterogeneous, players and prizes that include many existing contest models as special cases. We show that the equilibria of such contests are approximated by an appropriately defined set of incentive-compatible individually-rational single-agent mechanisms. This approach makes it possible to approximate the equilibria of contests whose exact equilibrium characterization is complicated, as well as the equilibria of contests for which there is no existing equilibrium characterization. design, welfare analysis, and comparative statics. This facilitates contest 1 *(abstract via OpenAlex)*

JMLR Other · 发表于 2016

LLORMA: Local Low-Rank Matrix Approximation

Joonseok Lee, Seungyeon Kim, Guy Lebanon, Yoram Singer, Samy Bengio

摘要

Matrix approximation is a common tool in recommendation systems, text mining, and computer vision. A prevalent assumption in constructing matrix approximations is that the partially observed matrix is low-rank. In this paper, we propose, analyze, and experiment with two procedures, one parallel and the other global, for constructing local matrix approximations. The two approaches approximate the observed matrix as a weighted sum of low-rank matrices. These matrices are limited to a local region of the observed matrix. We analyze the accuracy of the proposed local low-rank modeling. Our experiments show improvements in prediction accuracy over classical approaches for recommendation tasks.

JMLR Optimization & Learning Theory · 发表于 2016

LIBMF: A Library for Parallel Matrix Factorization in Shared-memory Systems

Wei-Sheng Chin, Bo-Wen Yuan, Meng-Yuan Yang, Yong Zhuang, Yu-Chin Juan, Chih-Jen Lin

摘要

Matrix factorization (MF) plays a key role in many applications such as recommender systems and computer vision, but MF may take long running time for handling large matrices commonly seen in the big data era. Many parallel techniques have been proposed to reduce the running time, but few parallel MF packages are available. Therefore, we present an open source library, LIBMF, based on recent advances of parallel MF for shared-memory systems. LIBMF includes easy-to-use command-line tools, interfaces to C/C++ languages, and comprehensive documentation. Our experiments demonstrate that LIBMF outperforms state of the art packages. LIBMF is BSD-licensed, so users can freely use, modify, and redistribute the code.

JMLR High-Dimensional · 发表于 2016

L1-Regularized Least Squares for Support Recovery of High Dimensional Single Index Models with Gaussian Designs

Matey Neykov, Jun S. Liu, Tianxi Cai

摘要

It is known that for a certain class of single index models (SIMs) $Y = f(X_{p \times 1}^\top\beta_0, \varepsilon)$, support recovery is impossible when $X \sim \mathcal{N}(0, I_{p \times p})$ and a model complexity adjusted sample size is below a critical threshold. Recently, optimal algorithms based on Sliced Inverse Regression (SIR) were suggested. These algorithms work provably under the assumption that the design $X$ comes from an i.i.d. Gaussian distribution. In the present paper we analyze algorithms based on covariance screening and least squares with $L_1$ penalization (i.e. LASSO) and demonstrate that they can also enjoy optimal (up to a scalar) rescaled sample size in terms of support recovery, albeit under slightly different assumptions on $f$ and $\varepsilon$ compared to the SIR based algorithms. Furthermore, we show more generally, that LASSO succeeds in recovering the signed support of $\beta_0$ if $X \sim \mathcal{N}(0, \Sigma)$, and the covariance $\Sigma$ satisfies the irrepresentable condition. Our work extends existing results on the support recovery of LASSO for the linear model, to a more general class of SIMs.

JMLR Optimization & Learning Theory Bayesian Methods Machine Learning · 发表于 2016

Knowledge Matters: Importance of Prior Information for Optimization

Çağlar Gülçehre, Yoshua Bengio

摘要

We explored the effect of introducing prior knowledge into the intermediate level of deep supervised neural networks on two tasks. On a task we designed, all black-box state-of-the-art machine learning algorithms which we tested, failed to generalize well. We motivate our work from the hypothesis that, there is a training barrier involved in the nature of such tasks, and that humans learn useful intermediate concepts from other individuals by using a form of supervision or guidance using a curriculum. Our results provide a positive evidence in favor of this hypothesis. In our experiments, we trained a two- tiered MLP architecture on a dataset for which each input image contains three sprites, and the binary target class is $1$ if all of three shapes belong to the same category and otherwise the class is $0$. In terms of generalization, black-box machine learning algorithms could not perform better than chance on this task. Standard deep supervised neural networks also failed to generalize. However, using a particular structure and guiding the learner by providing intermediate targets in the form of intermediate concepts (the presence of each object) allowed us to solve the task efficiently. We obtained much better than chance, but imperfect results by exploring different architectures and optimization variants. This observation might be an indication of optimization difficulty when the neural network trained without hints on this task. We hypothesize that the learning difficulty is due to the composition of two highly non-linear tasks. Our findings are also consistent with the hypotheses on cultural learning inspired by the observations of training of neural networks sometimes getting stuck, even though good solutions exist, both in terms of training and generalization error.

JMLR Nonparametric & Semiparametric · 发表于 2016

Kernel Mean Shrinkage Estimators

Krikamol Muandet, Bharath Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Bernhard Schölkopf

摘要

A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in RKHSs. Given a finite sample, an empirical average has been used commonly as a standard estimator of the true kernel mean. Despite a widespread use of this estimator, we show that it can be improved thanks to the well-known Stein phenomenon. We propose a new family of estimators called kernel mean shrinkage estimators (KMSEs), which benefit from both theoretical justifications and good empirical performance. The results demonstrate that the proposed estimators outperform the standard one, especially in a "large $d$, small $n$" paradigm.

JMLR Optimization & Learning Theory Nonparametric & Semiparametric · 发表于 2016

Kernel Estimation and Model Combination in A Bandit Problem with Covariates

Wei Qian, Yuhong Yang

摘要

Multi-armed bandit problem is an important optimization game that requires an exploration-exploitation tradeoff to achieve optimal total reward. Motivated from industrial applications such as online advertising and clinical research, we consider a setting where the rewards of bandit machines are associated with covariates, and the accurate estimation of the corresponding mean reward functions plays an important role in the performance of allocation rules. Under a flexible problem setup, we establish asymptotic strong consistency and perform a finite- time regret analysis for a sequential randomized allocation strategy based on kernel estimation. In addition, since many nonparametric and parametric methods in supervised learning may be applied to estimating the mean reward functions but guidance on how to choose among them is generally unavailable, we propose a model combining allocation strategy for adaptive performance. Simulations and a real data evaluation are conducted to illustrate the performance of the proposed allocation strategy.

JMLR High-Dimensional · 发表于 2016

Jointly Informative Feature Selection Made Tractable by Gaussian Modeling

Leonidas Lefakis, François Fleuret

摘要

We address the problem of selecting groups of jointly informative, continuous, features in the context of classification and propose several novel criteria for performing this selection. The proposed class of methods is based on combining a Gaussian modeling of the feature responses with derived bounds on and approximations to their mutual information with the class label. Furthermore, specific algorithmic implementations of these criteria are presented which reduce the computational complexity of the proposed feature selection algorithms by up to two-orders of magnitude. Consequently we show that feature selection based on the joint mutual information of features and class label is in fact tractable; this runs contrary to prior works that largely depend on marginal quantities. An empirical evaluation using several types of classifiers on multiple data sets show that this class of methods outperforms state-of-the-art baselines, both in terms of speed and classification accuracy.

JMLR Networks & Graphs High-Dimensional · 发表于 2016

Joint Structural Estimation of Multiple Graphical Models

Jing Ma, George Michailidis

摘要

Gaussian graphical models capture dependence relationships between random variables through the pattern of nonzero elements in the corresponding inverse covariance matrices. To date, there has been a large body of literature on both computational methods and analytical results on the estimation of a single graphical model. However, in many application domains, one has to estimate several related graphical models, a problem that has also received attention in the literature. The available approaches usually assume that all graphical models are globally related. On the other hand, in many settings different relationships between subsets of the node sets exist between different graphical models. We develop methodology that jointly estimates multiple Gaussian graphical models, assuming that there exists prior information on how they are structurally related. For many applications, such information is available from external data sources. The proposed method consists of first applying neighborhood selection with a group lasso penalty to obtain edge sets of the graphs, and a maximum likelihood refit for estimating the nonzero entries in the inverse covariance matrices. We establish consistency of the proposed method for sparse high-dimensional Gaussian graphical models and examine its performance using simulation experiments. Applications to a climate data set and a breast cancer data set are also discussed.

JMLR Other · 发表于 2016

JCLAL: A Java Framework for Active Learning

Oscar Reyes, Eduardo Pérez, María del Carmen Rodríguez-Hernández, Habib M. Fardoun, Sebastián Ventura

摘要

Active Learning has become an important area of research owing to the increasing number of real-world problems which contain labelled and unlabelled examples at the same time. JCLAL is a Java Class Library for Active Learning which has an architecture that follows strong principles of object-oriented design. It is easy to use, and it allows the developers to adapt, modify and extend the framework according to their needs. The library offers a variety of active learning methods that have been proposed in the literature. The software is available under the GPL license.

JMLR Other · 发表于 2016

Iterative Regularization for Learning with Convex Loss Functions

Junhong Lin, Lorenzo Rosasco, Ding-Xuan Zhou

摘要

We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method. Unlike other regularization approaches, in iterative regularization no constraint or penalization is considered, and generalization is achieved by (early) stopping an empirical iteration. We consider a nonparametric setting, in the framework of reproducing kernel Hilbert spaces, and prove consistency and finite sample bounds on the excess risk under general regularity conditions. Our study provides a new class of efficient regularized learning algorithms and gives insights on the interplay between statistics and optimization in machine learning.

JMLR Other · 发表于 2016

Iterative Hessian Sketch: Fast and Accurate Solution Approximation for Constrained Least-Squares

Mert Pilanci, Martin J. Wainwright

摘要

We study randomized sketching methods for approximately solving least-squares problem with a general convex constraint. The quality of a least-squares approximation can be assessed in different ways: either in terms of the value of the quadratic objective function (cost approximation), or in terms of some distance measure between the approximate minimizer and the true minimizer (solution approximation). Focusing on the latter criterion, our first main result provides a general lower bound on any randomized method that sketches both the data matrix and vector in a least-squares problem; as a surprising consequence, the most widely used least-squares sketch is sub-optimal for solution approximation. We then present a new method known as the iterative Hessian sketch, and show that it can be used to obtain approximations to the original least-squares problem using a projection dimension proportional to the statistical complexity of the least-squares minimizer, and a logarithmic number of iterations. We illustrate our general theory with simulations for both unconstrained and constrained versions of least-squares, including $\ell_1$-regularization and nuclear norm constraints. We also numerically demonstrate the practicality of our approach in a real face expression classification experiment.

JMLR Machine Learning · 发表于 2016

Interleaved Text/Image Deep Mining on a Large-Scale Radiology Database for Automated Image Interpretation

Hoo-Chang Shin, Le Lu, Lauren Kim, Ari Seff, Jianhua Yao, Ronald M. Summers

摘要

Despite tremendous progress in computer vision, there has not been an attempt to apply machine learning on very large-scale medical image databases. We present an interleaved text/image deep learning system to extract and mine the semantic interactions of radiology images and reports from a national research hospital's Picture Archiving and Communication System. With natural language processing, we mine a collection of $\sim$216K representative two-dimensional images selected by clinicians for diagnostic reference and match the images with their descriptions in an automated manner. We then employ a weakly supervised approach using all of our available data to build models for generating approximate interpretations of patient images. Finally, we demonstrate a more strictly supervised approach to detect the presence and absence of a number of frequent disease types, providing more specific interpretations of patient scans. A relatively small amount of data is used for this part, due to the challenge in gathering quality labels from large raw text data. Our work shows the feasibility of large-scale learning and prediction in electronic patient records available in most modern clinical institutions. It also demonstrates the trade-offs to consider in designing machine learning systems for analyzing large medical data.

JMLR Latent Variables & Mixtures Longitudinal Data · 发表于 2016

Integrative Analysis using Coupled Latent Variable Models for Individualizing Prognoses

Peter Schulam, Suchi Saria

摘要

Complex chronic diseases (e.g., autism, lupus, and Parkinson's) are remarkably heterogeneous across individuals. This heterogeneity makes treatment difficult for caregivers because they cannot accurately predict the way in which the disease will progress in order to guide treatment decisions. Therefore, tools that help to predict the trajectory of these complex chronic diseases can help to improve the quality of health care. To build such tools, we can leverage clinical markers that are collected at baseline when a patient first presents and longitudinally over time during follow-up visits. Because complex chronic diseases are typically systemic, the longitudinal markers often track disease progression in multiple organ systems. In this paper, our goal is to predict a function of time that models the future trajectory of a single target clinical marker tracking a disease process of interest. We want to make these predictions using the histories of many related clinical markers as input. Our proposed solution tackles several key challenges. First, we can easily handle irregularly and sparsely sampled markers, which are standard in clinical data. Second, the number of parameters and the computational complexity of learning our model grows linearly in the number of marker types included in the model. This makes our approach applicable to diseases where many different markers are recorded over time. Finally, our model accounts for latent factors influencing disease expression, whereas standard regression models rely on observed features alone to explain variability. Moreover, our approach can be applied dynamically in continous- time and updates its predictions as soon as any new data is available. We apply our approach to the problem of predicting lung disease trajectories in scleroderma, a complex autoimmune disease. We show that our model improves over state-of-the-art baselines in predictive accuracy and we provide a qualitative analysis of our model's output. Finally, the variability of disease presentation in scleroderma makes clinical trial recruitment challenging. We show that a prognostic tool that integrates multiple types of routinely collected longitudinal data can be used to identify individuals at greatest risk of rapid progression and to target trial recruitment.

JMLR Other · 发表于 2016

Integrated Common Sense Learning and Planning in POMDPs

Brendan Juba

摘要

We formulate a new variant of the problem of planning in an unknown environment, for which we can provide algorithms with reasonable theoretical guarantees in spite of large state spaces and time horizons, partial observability, and complex dynamics. In this variant, an agent is given a collection of example traces produced by a reference policy, which may, for example, capture the agent's past behavior. The agent is (only) asked to find policies that are supported by regularities in the dynamics that are observable on these example traces. We describe an efficient algorithm that uses such common sense knowledge reflected in the example traces to construct decision tree policies for goal-oriented factored POMDPs. More precisely, our algorithm (provably) succeeds at finding a policy for a given input goal when (1) there is a CNF that is almost always observed satisfied on the traces of the POMDP, capturing a sufficient approximation of its dynamics and (2) for a decision tree policy of bounded complexity, there exist small- space resolution proofs that the goal is achieved on each branch using the aforementioned CNF capturing the common sense rules. Such a CNF always exists for noisy STRIPS domains, for example. Our results thus essentially establish that the possession of a suitable exploration policy for collecting the necessary examples is the fundamental obstacle to learning to act in such environments.

Econometrica Econometrics & Economics Spatial & Time Series · 发表于 2016

Insider Trading, Stochastic Liquidity, and Equilibrium Prices

Pierre Collin-Dufresne, Vyacheslav Fos

摘要

We extend Kyle's (1985) model of insider trading to the case where noise trading volatility follows a general stochastic process. We determine conditions under which, in equilibrium, price impact and price volatility are both stochastic, driven by shocks to uninformed volume even though the fundamental value is constant. The volatility of price volatility appears 'excessive' because insiders choose to trade more aggressively (and thus more information is revealed) when uninformed volume is higher and price impact is lower. This generates a positive relation between price volatility and trading volume, giving rise to an endogenous subordinate stochastic process for prices. *(abstract via OpenAlex)*

JMLR Nonparametric & Semiparametric · 发表于 2016

Input Output Kernel Regression: Supervised and Semi-Supervised Structured Output Prediction with Operator-Valued Kernels

Céline Brouard, Marie Szafranski, Florence d'Alché-Buc

摘要

In this paper, we introduce a novel approach, called Input Output Kernel Regression (IOKR), for learning mappings between structured inputs and structured outputs. The approach belongs to the family of Output Kernel Regression methods devoted to regression in feature space endowed with some output kernel. In order to take into account structure in input data and benefit from kernels in the input space as well, we use the Reproducing Kernel Hilbert Space theory for vector-valued functions. We first recall the ridge solution for supervised learning and then study the regularized hinge loss-based solution used in Maximum Margin Regression. Both models are also developed in the context of semi-supervised setting. In addition we derive an extension of Generalized Cross Validation for model selection in the case of the least-square model. Finally we show the versatility of the IOKR framework on two different problems: link prediction seen as a structured output problem and multi-task regression seen as a multiple and interdependent output problem. Eventually, we present a set of detailed numerical results that shows the relevance of the method on these two tasks.

Econometrica Other · 发表于 2016

Information in Tender Offers With a Large Shareholder

Mehmet Ekmekci, Nenad Kos

摘要

We study takeovers of firms whose ownership structure is a mixture of minority block-holders and small shareholders. We show that the combination of dispersed private information on the side of small shareholders and the presence of a large shareholder can facilitate profitable takeovers. Furthermore, our analysis implies that even if some model of takeovers predicts a profit for the raider, for example due to private benefits, the profit will be underestimated unless the large shareholder and the dispersion of information among the small shareholders are modeled. *(abstract via OpenAlex)*

Econometrica Econometrics & Economics · 发表于 2016

Inefficient Investment Waves

Zhiguo He, Péter Kondor

摘要

We show that firms’ individually optimal liquidity management results in socially inefficient boom-and-bust patterns. Financially constrained firms decide on the level of their liquid resources facing cash-flow shocks and time-varying investment opportunities. Firms; liquidity management decisions generate simoultaneous waves in aggregate cash holdings, in market value of liquidity and in investment even if technology remains constant, consistently with firm-level and aggregate evidence. These investment waves are not constrained efficient in general, because the social and private value of liquidity differs. The resulting pecuniary externality affect incentives differentially depending on the state of the economy. There is often overinvestment in booms and underinvestment in recessions. In general, policies targeted to raise prices in recessions to mitigate underinvestment, make overinvestment in booms worse. However, a well designed price-support policy will increase welfare both in booms and in recessions. *(abstract via OpenAlex)*

Econometrica Other · 发表于 2016

Individual Heterogeneity and Average Welfare

Jerry A. Hausman, Whitney K. Newey

摘要

Individual heterogeneity is an important source of variation in demand. Allowing for general heterogeneity is needed for correct welfare comparisons. We consider general heterogenous demand where preferences and linear budget sets are statistically independent. We find that the dimension of heterogeneity and the individual demand functions are not identified. We also find that the exact consumer surplus of a price change, averaged across individuals, is not identified, motivating bounds analysis. We use bounds on income effects to derive relatively simple bounds on the average surplus, including for discrete/continuous choice. We also sketch an approach to bounding surplus that does not use income effect bounds. We apply the results with income effect bounds to gasoline demand. We find little sensitivity to the income effect bounds in this application. *(abstract via OpenAlex)*

JMLR Bayesian Methods Networks & Graphs · 发表于 2016

Improving Structure MCMC for Bayesian Networks through Markov Blanket Resampling

Chengwei Su, Mark E. Borsuk

摘要

Algorithms for inferring the structure of Bayesian networks from data have become an increasingly popular method for uncovering the direct and indirect influences among variables in complex systems. A Bayesian approach to structure learning uses posterior probabilities to quantify the strength with which the data and prior knowledge jointly support each possible graph feature. Existing Markov Chain Monte Carlo (MCMC) algorithms for estimating these posterior probabilities are slow in mixing and convergence, especially for large networks. We present a novel Markov blanket resampling (MBR) scheme that intermittently reconstructs the Markov blanket of nodes, thus allowing the sampler to more effectively traverse low-probability regions between local maxima. As we can derive the complementary forward and backward directions of the MBR proposal distribution, the Metropolis-Hastings algorithm can be used to account for any asymmetries in these proposals. Experiments across a range of network sizes show that the MBR scheme outperforms other state- of-the-art algorithms, both in terms of learning performance and convergence rate. In particular, MBR achieves better learning performance than the other algorithms when the number of observations is relatively small and faster convergence when the number of variables in the network is large.

JMLR Optimization & Learning Theory · 发表于 2016

Importance Weighting Without Importance Weights: An Efficient Algorithm for Combinatorial Semi-Bandits

Gergely Neu, Gábor Bartók

摘要

We propose a sample-efficient alternative for importance weighting for situations where one only has sample access to the probability distribution that generates the observations. Our new method, called Geometric Resampling (GR), is described and analyzed in the context of online combinatorial optimization under semi-bandit feedback, where a learner sequentially selects its actions from a combinatorial decision set so as to minimize its cumulative loss. In particular, we show that the well-known Follow-the-Perturbed-Leader (FPL) prediction method coupled with Geometric Resampling yields the first computationally efficient reduction from offline to online optimization in this setting. We provide a thorough theoretical analysis for the resulting algorithm, showing that its performance is on par with previous, inefficient solutions. Our main contribution is showing that, despite the relatively large variance induced by the GR procedure, our performance guarantees hold with high probability rather than only in expectation. As a side result, we also improve the best known regret bounds for FPL in online combinatorial optimization with full feedback, closing the perceived performance gap between FPL and exponential weights in this setting. (A preliminary version of this paper was published as Neu and Bartók (2013). Parts of this work were completed while Gergely Neu was with the SequeL team at INRIA Lille -- Nord Europe, France and Gábor Bartók was with the Department of Computer Science at ETH Zürich.)

Econometrica Nonparametric & Semiparametric Econometrics & Economics · 发表于 2016

IV Quantile Regression for Group-Level Treatments, With an Application to the Distributional Effects of Trade

Denis Chetverikov, Bradley Larsen, Christopher Palmer

摘要

We present a methodology for estimating the distributional effects of an endogenous treatment that varies at the group level when there are group-level unobservables, a quantile extension of Hausman and Taylor (1981). Standard quantile regression techniques are inconsistent in this setting, even if the treatment is exogenous. Using the Bahadur representation of quantile estimators, we derive weak conditions on the growth of the number of observations per group that are sufficient for consistency and asymptotic normality. Simulations confirm the superiority of this grouped instrumental variables quantile regression estimator to standard quantile regression. An empirical application finds that low-wage earners in the U.S. from 1990-2007 were significantly more affected by increased Chinese import competition than high-wage earners. We also illustrate the usefulness of the estimation approach with additional empirical examples from urban economics, labor, regulation, and empirical auctions. *(abstract via OpenAlex)*

JMLR Networks & Graphs Machine Learning High-Dimensional · 发表于 2016

Hybrid Orthogonal Projection and Estimation (HOPE): A New Framework to Learn Neural Networks

Shiliang Zhang, Hui Jiang, Lirong Dai

摘要

In this paper, we propose a novel model for high-dimensional data, called the Hybrid Orthogonal Projection and Estimation (HOPE) model, which combines a linear orthogonal projection and a finite mixture model under a unified generative modeling framework. The HOPE model itself can be learned unsupervised from unlabelled data based on the maximum likelihood estimation as well as discriminatively from labelled data. More interestingly, we have shown the proposed HOPE models are closely related to neural networks (NNs) in a sense that each hidden layer can be reformulated as a HOPE model. As a result, the HOPE framework can be used as a novel tool to probe why and how NNs work, more importantly, to learn NNs in either supervised or unsupervised ways. In this work, we have investigated the HOPE framework to learn NNs for several standard tasks, including image recognition on MNIST and speech recognition on TIMIT. Experimental results have shown that the HOPE framework yields significant performance gains over the current state-of-the-art methods in various types of NN learning problems, including unsupervised feature learning, supervised or semi-supervised learning.

JMLR Other · 发表于 2016

How to Center Deep Boltzmann Machines

Jan Melchior, Asja Fischer, Laurenz Wiskott

摘要

This work analyzes centered Restricted Boltzmann Machines (RBMs) and centered Deep Boltzmann Machines (DBMs), where centering is done by subtracting offset values from visible and hidden variables. We show analytically that (i) centered and normal Boltzmann Machines (BMs) and thus RBMs and DBMs are different parameterizations of the same model class, such that any normal BM/RBM/DBM can be transformed to an equivalent centered BM/RBM/DBM and vice versa, and that this equivalence generalizes to artificial neural networks in general, (ii) the expected performance of centered binary BMs/RBMs/DBMs is invariant under simultaneous flip of data and offsets, for any offset value in the range of zero to one, (iii) centering can be reformulated as a different update rule for normal BMs/RBMs/DBMs, and (iv) using the enhanced gradient is equivalent to setting the offset values to the average over model and data mean. Furthermore, we present numerical simulations suggesting that (i) optimal generative performance is achieved by subtracting mean values from visible as well as hidden variables, (ii) centered binary RBMs/DBMs reach significantly higher log-likelihood values than normal binary RBMs/DBMs, (iii) centering variants whose offsets depend on the model mean, like the enhanced gradient, suffer from severe divergence problems, (iv) learning is stabilized if an exponentially moving average over the batch means is used for the offset values instead of the current batch mean, which also prevents the enhanced gradient from severe divergence, (v) on a similar level of log-likelihood values centered binary RBMs/DBMs have smaller weights and bigger bias parameters than normal binary RBMs/DBMs, (vi) centering leads to an update direction that is closer to the natural gradient, which is extremely efficient for training as we show for small binary RBMs, (vii) centering eliminates the need for greedy layer-wise pre-training of DBMs, which often even deteriorates the results independently of whether centering is used or not, and (ix) centering is also beneficial for auto encoders.

JMLR Other · 发表于 2016

Hierarchical Relative Entropy Policy Search

Christian Daniel, Gerhard Neumann, Oliver Kroemer, Jan Peters

摘要

Many reinforcement learning (RL) tasks, especially in robotics, consist of multiple sub-tasks that are strongly structured. Such task structures can be exploited by incorporating hierarchical policies that consist of gating networks and sub-policies. However, this concept has only been partially explored for real world settings and complete methods, derived from first principles, are needed. Real world settings are challenging due to large and continuous state-action spaces that are prohibitive for exhaustive sampling methods. We define the problem of learning sub-policies in continuous state action spaces as finding a hierarchical policy that is composed of a high-level gating policy to select the low-level sub-policies for execution by the agent. In order to efficiently share experience with all sub-policies, also called inter-policy learning, we treat these sub-policies as latent variables which allows for distribution of the update information between the sub-policies. We present three different variants of our algorithm, designed to be suitable for a wide variety of real world robot learning tasks and evaluate our algorithms in two real robot learning scenarios as well as several simulations and comparisons.

JMLR Bayesian Methods Networks & Graphs · 发表于 2016

Herded Gibbs Sampling

Yutian Chen, Luke Bornn, Nando de Freitas, Mareija Eskelin, Jing Fang, Max Welling

摘要

The Gibbs sampler is one of the most popular algorithms for inference in statistical models. In this paper, we introduce a herding variant of this algorithm, called herded Gibbs, that is entirely deterministic. We prove that herded Gibbs has an $O(1/T)$ convergence rate for models with independent variables and for fully connected probabilistic graphical models. Herded Gibbs is shown to outperform Gibbs in the tasks of image denoising with MRFs and named entity recognition with CRFs. However, the convergence for herded Gibbs for sparsely connected probabilistic graphical models is still an open problem.

JMLR Other · 发表于 2016

Harry: A Tool for Measuring String Similarity

Konrad Rieck, Christian Wressnegger

摘要

Comparing strings and assessing their similarity is a basic operation in many application domains of machine learning, such as in information retrieval, natural language processing and bioinformatics. The practitioner can choose from a large variety of available similarity measures for this task, each emphasizing different aspects of the string data. In this article, we present Harry, a small tool specifically designed for measuring the similarity of strings. Harry implements over 20 similarity measures, including common string distances and string kernels, such as the Levenshtein distance and the Subsequence kernel. The tool has been designed with efficiency in mind and allows for multi-threaded as well as distributed computing, enabling the analysis of large data sets of strings. Harry supports common data formats and thus can interface with analysis environments, such as Matlab, Pylab and Weka.

JMLR High-Dimensional · 发表于 2016

Guarding against Spurious Discoveries in High Dimensions

Jianqing Fan, Wen-Xin Zhou

摘要

Many data mining and statistical machine learning algorithms have been developed to select a subset of covariates to associate with a response variable. Spurious discoveries can easily arise in high-dimensional data analysis due to enormous possibilities of such selections. How can we know statistically our discoveries better than those by chance? In this paper, we define a measure of goodness of spurious fit, which shows how good a response variable can be fitted by an optimally selected subset of covariates under the null model, and propose a simple and effective LAMM algorithm to compute it. It coincides with the maximum spurious correlation for linear models and can be regarded as a generalized maximum spurious correlation. We derive the asymptotic distribution of such goodness of spurious fit for generalized linear models and $L_1$ regression. Such an asymptotic distribution depends on the sample size, ambient dimension, the number of variables used in the fit, and the covariance information. It can be consistently estimated by multiplier bootstrapping and used as a benchmark to guard against spurious discoveries. It can also be applied to model selection, which considers only candidate models with goodness of fits better than those by spurious fits. The theory and method are convincingly illustrated by simulated examples and an application to the binary outcomes from German Neuroblastoma Trials.

JMLR Other · 发表于 2016

Gradients Weights improve Regression and Classification

Samory Kpotufe, Abdeslam Boularias, Thomas Schultz, Kyoungok Kim

摘要

In regression problems over $\mathbb{R}^d$, the unknown function $f$ often varies more in some coordinates than in others. We show that weighting each coordinate $i$ according to an estimate of the variation of $f$ along coordinate $i$ -- e.g. the $L_1$ norm of the $i$th-directional derivative of $f$ -- is an efficient way to significantly improve the performance of distance-based regressors such as kernel and $k$-NN regressors. The approach, termed Gradient Weighting (GW), consists of a first pass regression estimate $f_n$ which serves to evaluate the directional derivatives of $f$, and a second-pass regression estimate on the re-weighted data. The GW approach can be instantiated for both regression and classification, and is grounded in strong theoretical principles having to do with the way regression bias and variance are affected by a generic feature-weighting scheme. These theoretical principles provide further technical foundation for some existing feature-weighting heuristics that have proved successful in practice. We propose a simple estimator of these derivative norms and prove its consistency. The proposed estimator computes efficiently and easily extends to run online. We then derive a classification version of the GW approach which evaluates on real-worlds datasets with as much success as its regression counterpart.

Econometrica Spatial & Time Series Nonparametric & Semiparametric · 发表于 2016

Generalized Method of Integrated Moments for High-Frequency Data

Jia Li, Dacheng Xiu

摘要

We propose a semiparametric two-step inference procedure for a finite-dimensional parameter based on moment conditions constructed from high-frequency data. The population moment conditions take the form of temporally integrated functionals of state-variable processes that include the latent stochastic volatility process of an asset. In the first step, we nonparametrically recover the volatility path from high-frequency asset returns. The nonparametric volatility estimator is then used to form sample moment functions in the second-step GMM estimation, which requires the correction of a high-order nonlinearity bias from the first step. We show that the proposed estimator is consistent and asymptotically mixed Gaussian and propose a consistent estimator for the conditional asymptotic variance. We also construct a Bierens-type consistent specification test. These infill asymptotic results are based on a novel empirical-process-type theory for general integrated functionals of noisy semimartingale processes. *(abstract via OpenAlex)*

JMLR Optimization & Learning Theory · 发表于 2016

GenSVM: A Generalized Multiclass Support Vector Machine

Gerrit J.J. van den Burg, Patrick J.F. Groenen

摘要

Traditional extensions of the binary support vector machine (SVM) to multiclass problems are either heuristics or require solving a large dual optimization problem. Here, a generalized multiclass SVM is proposed called GenSVM. In this method classification boundaries for a $K$-class problem are constructed in a $(K-1)$-dimensional space using a simplex encoding. Additionally, several different weightings of the misclassification errors are incorporated in the loss function, such that it generalizes three existing multiclass SVMs through a single optimization problem. An iterative majorization algorithm is derived that solves the optimization problem without the need of a dual formulation. This algorithm has the advantage that it can use warm starts during cross validation and during a grid search, which significantly speeds up the training phase. Rigorous numerical experiments compare linear GenSVM with seven existing multiclass SVMs on both small and large data sets. These comparisons show that the proposed method is competitive with existing methods in both predictive accuracy and training time, and that it significantly outperforms several existing methods on these criteria.

JMLR Optimization & Learning Theory High-Dimensional · 发表于 2016

Gains and Losses are Fundamentally Different in Regret Minimization: The Sparse Case

Joon Kwon, Vianney Perchet

摘要

We demonstrate that, in the classical non-stochastic regret minimization problem with $d$ decisions, gains and losses to be respectively maximized or minimized are fundamentally different. Indeed, by considering the additional sparsity assumption (at each stage, at most $s$ decisions incur a nonzero outcome), we derive optimal regret bounds of different orders. Specifically, with gains, we obtain an optimal regret guarantee after $T$ stages of order $\sqrt{T\log s}$, so the classical dependency in the dimension is replaced by the sparsity size. With losses, we provide matching upper and lower bounds of order $\sqrt{Ts\log(d)/d}$, which is decreasing in $d$. Eventually, we also study the bandit setting, and obtain an upper bound of order $\sqrt{Ts\log (d/s)}$ when outcomes are losses. This bound is proven to be optimal up to the logarithmic factor $\sqrt{\log(d/s)}$.

JMLR High-Dimensional Latent Variables & Mixtures Meta-Analysis & Data Integration · 发表于 2016

Fused Lasso Approach in Regression Coefficients Clustering -- Learning Parameter Heterogeneity in Data Integration

Lu Tang, Peter X.K. Song

摘要

As data sets of related studies become more easily accessible, combining data sets of similar studies is often undertaken in practice to achieve a larger sample size and higher power. A major challenge arising from data integration pertains to data heterogeneity in terms of study population, study design, or study coordination. Ignoring such heterogeneity in data analysis may result in biased estimation and misleading inference. Traditional techniques of remedy to data heterogeneity include the use of interactions and random effects, which are inferior to achieving desirable statistical power or providing a meaningful interpretation, especially when a large number of smaller data sets are combined. In this paper, we propose a regularized fusion method that allows us to identify and merge inter-study homogeneous parameter clusters in regression analysis, without the use of hypothesis testing approach. Using the fused lasso, we establish a computationally efficient procedure to deal with large-scale integrated data. Incorporating the estimated parameter ordering in the fused lasso facilitates computing speed with no loss of statistical power. We conduct extensive simulation studies and provide an application example to demonstrate the performance of the new method with a comparison to the conventional methods.

Econometrica Other · 发表于 2016

From Bottom of the Barrel to Cream of the Crop: Sequential Screening With Positive Selection

Jean Tirole

摘要

In a number of interesting environments, dynamic screening involves positive selection: in contrast with Coasian dynamics, only the most motivated remain over time. The paper provides conditions under which the principal's commitment optimum is time consistent and uses this result to derive testable predictions under permanent or transient shocks. It also identifies environments in which time consistency does not hold despite positive selection, and yet simple equilibrium characterizations can be obtained. *(abstract via OpenAlex)*

Econometrica Econometrics & Economics Genetics & Omics · 发表于 2016

Financial Health Economics

Ralph S. J. Koijen, Tomas J. Philipson, Harald Uhlig

摘要

We provide a theoretical and empirical analysis of the link between financial and real health care markets. This link is important as financial returns drive investment in medical research and development (R&D), which in turn, affects real spending growth. We document a "medical innovation premium" of 4-6% annually for equity returns of firms in the health care sector. We interpret this premium as compensating investors for government-induced profit risk, and we provide supportive evidence for this hypothesis through company filings and abnormal return patterns surrounding threats of government intervention. We quantify the implications of the premium for the growth in real health care spending by calibrating our model to match historical trends, predicting a long run share of 34%. Policies that had removed government risk would have led to more than a doubling of medical R&D and would have increased the current share of health care spending by nearly 4% of GDP. *(abstract via OpenAlex)*

Econometrica Econometrics & Economics · 发表于 2016

Female Labor Supply, Human Capital, and Welfare Reform

Richard Blundell, Monica Costa Dias, Costas Meghir, Jonathan Shaw

摘要

We estimate a dynamic model of employment, human capital accumulation — including education, and savings for women in the UK, exploiting policy changes. We analyze both the incentive effects and the welfare implications of tax credits and income support programs and we account for their insurance value. We find important incentive effects on education choice and labor supply, with single mothers having the most elastic labor supply. Returns to experience increase with education, but experience only accumulates when in full-time employment. Finally, marginal increases in tax credits are preferred to equally costly income support or to tax cuts. *(abstract via OpenAlex)*

JMLR Other · 发表于 2016

Feature-Level Domain Adaptation

Wouter M. Kouw, Laurens J.P. van der Maaten, Jesse H. Krijthe, Marco Loog

摘要

Domain adaptation is the supervised learning setting in which the training and test data are sampled from different distributions: training data is sampled from a source domain, whilst test data is sampled from a target domain. This paper proposes and studies an approach, called feature-level domain adaptation (FLDA), that models the dependence between the two domains by means of a feature-level transfer model that is trained to describe the transfer from source to target domain. Subsequently, we train a domain-adapted classifier by minimizing the expected loss under the resulting transfer model. For linear classifiers and a large family of loss functions and transfer models, this expected loss can be computed or approximated analytically, and minimized efficiently. Our empirical evaluation of FLDA focuses on problems comprising binary and count data in which the transfer can be naturally modeled via a dropout distribution, which allows the classifier to adapt to differences in the marginal probability of features in the source and the target domain. Our experiments on several real- world problems show that FLDA performs on par with state- of- the-art domain-adaptation techniques.

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 页