2016 年的期刊文章 · 第 28 页

共 1345 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2016 到 2016。

按期刊、按主题看 →

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 页

JMLR Longitudinal Data Latent Variables & Mixtures · 发表于 2016

The Factorized Self-Controlled Case Series Method: An Approach for Estimating the Effects of Many Drugs on Many Outcomes

Ramin Moghaddass, Cynthia Rudin, David Madigan

摘要

We provide a hierarchical Bayesian model for estimating the effects of transient drug exposures on a collection of health outcomes, where the effects of all drugs on all outcomes are estimated simultaneously. The method possesses properties that allow it to handle important challenges of dealing with large- scale longitudinal observational databases. In particular, this model is a generalization of the self-controlled case series (SCCS) method, meaning that certain patient specific baseline rates never need to be estimated. Further, this model is formulated with layers of latent factors, which substantially reduces the number of parameters and helps with interpretability by illuminating latent classes of drugs and outcomes. We believe our work is the first to consider multivariate SCCS (in the sense of multiple outcomes) and is the first to couple latent factor analysis with SCCS. We demonstrate the approach by estimating the effects of various time-sensitive insulin treatments for diabetes.

Econometrica Econometrics & Economics · 发表于 2016

The Effect of Changes in Risk Attitude on Strategic Behavior

Jonathan Weinstein

摘要

We study families of normal-form games with fixed preferences over pure action profiles but varied preferences over lotteries. That is, we subject players' utilities to monotone but nonlinear transformations and examine changes in the rationalizable set and set of equilibria. Among our results: The rationalizable set always grows under concave transformations (risk aversion) and shrinks under convex transformations (risk love). The rationalizable set reaches an upper bound under extreme risk aversion, and lower bound under risk love, and both of these bounds are characterized by elimination processes. For generic two-player games, under extreme risk love or aversion, all Nash equilibria are close to pure and the limiting set of equilibria can be described using preferences over pure action profiles. [web URL: http://onlinelibrary.wiley.com/doi/10.3982/ECTA13948/abstract] *(abstract via OpenAlex)*

Econometrica Other · 发表于 2016

The Dynamics of Inequality

Xavier Gabaix, Jean-Michel Lasry, Pierre-Louis Lions, Benjamin Moll

摘要

The past forty years have seen a rapid rise in top income inequality in the United States. While there is a large number of existing theories of the Pareto tail of the long-run income distributions, almost none of these address the fast rise in top inequality observed in the data. We show that standard theories, which build on a random growth mechanism, generate transition dynamics that are too slow relative to those observed in the data. We then suggest two parsimonious deviations from the canonical model that can explain such changes: “scale dependence” that may arise from changes in skill prices, and “type dependence,” that is, the presence of some “high-growth types.” These deviations are consistent with theories in which the increase in top income inequality is driven by the rise of “superstar” entrepreneurs or managers. *(abstract via OpenAlex)*

JMLR High-Dimensional Clinical Trials · 发表于 2016

The Constrained Dantzig Selector with Enhanced Consistency

Yinfei Kong, Zemin Zheng, Jinchi Lv

摘要

The Dantzig selector has received popularity for many applications such as compressed sensing and sparse modeling, thanks to its computational efficiency as a linear programming problem and its nice sampling properties. Existing results show that it can recover sparse signals mimicking the accuracy of the ideal procedure, up to a logarithmic factor of the dimensionality. Such a factor has been shown to hold for many regularization methods. An important question is whether this factor can be reduced to a logarithmic factor of the sample size in ultra-high dimensions under mild regularity conditions. To provide an affirmative answer, in this paper we suggest the constrained Dantzig selector, which has more flexible constraints and parameter space. We prove that the suggested method can achieve convergence rates within a logarithmic factor of the sample size of the oracle rates and improved sparsity, under a fairly weak assumption on the signal strength. Such improvement is significant in ultra-high dimensions. This method can be implemented efficiently through sequential linear programming. Numerical studies confirm that the sample size needed for a certain level of accuracy in these problems can be much reduced.

JMLR Other · 发表于 2016

The Benefit of Multitask Representation Learning

Andreas Maurer, Massimiliano Pontil, Bernardino Romera-Paredes

摘要

We discuss a general method to learn data representations from multiple tasks. We provide a justification for this method in both settings of multitask learning and learning-to-learn. The method is illustrated in detail in the special case of linear feature learning. Conditions on the theoretical advantage offered by multitask representation learning over independent task learning are established. In particular, focusing on the important example of half-space learning, we derive the regime in which multitask representation learning is beneficial over independent task learning, as a function of the sample size, the number of tasks and the intrinsic data dimensionality. Other potential applications of our results include multitask feature learning in reproducing kernel Hilbert spaces and multilayer, deep networks.

JMLR Networks & Graphs Machine Learning · 发表于 2016

The Asymptotic Performance of Linear Echo State Neural Networks

Romain Couillet, Gilles Wainrib, Harry Sevi, Hafiz Tiomoko Ali

摘要

In this article, a study of the mean-square error (MSE) performance of linear echo-state neural networks is performed, both for training and testing tasks. Considering the realistic setting of noise present at the network nodes, we derive deterministic equivalents for the aforementioned MSE in the limit where the number of input data $T$ and network size $n$ both grow large. Specializing then the network connectivity matrix to specific random settings, we further obtain simple formulas that provide new insights on the performance of such networks.

JMLR Other · 发表于 2016

Synergy of Monotonic Rules

Vladimir Vapnik, Rauf Izmailov

摘要

This article describes a method for constructing a special rule (we call it synergy rule) that uses as its input information the outputs (scores) of several monotonic rules which solve the same pattern recognition problem. As an example of scores of such monotonic rules we consider here scores of SVM classifiers.

JMLR Survival Analysis · 发表于 2016

Support Vector Hazards Machine: A Counting Process Framework for Learning Risk Scores for Censored Outcomes

Yuanjia Wang, Tianle Chen, Donglin Zeng

摘要

Learning risk scores to predict dichotomous or continuous outcomes using machine learning approaches has been studied extensively. However, how to learn risk scores for time-to-event outcomes subject to right censoring has received little attention until recently. Existing approaches rely on inverse probability weighting or rank-based regression, which may be inefficient. In this paper, we develop a new support vector hazards machine (SVHM) approach to predict censored outcomes. Our method is based on predicting the counting process associated with the time-to-event outcomes among subjects at risk via a series of support vector machines. Introducing counting processes to represent time-to-event data leads to a connection between support vector machines in supervised learning and hazards regression in standard survival analysis. To account for different at risk populations at observed event times, a time-varying offset is used in estimating risk scores. The resulting optimization is a convex quadratic programming problem that can easily incorporate non-linearity using kernel trick. We demonstrate an interesting link from the profiled empirical risk function of SVHM to the Cox partial likelihood. We then formally show that SVHM is optimal in discriminating covariate-specific hazard function from population average hazard function, and establish the consistency and learning rate of the predicted risk using the estimated risk scores. Simulation studies show improved prediction accuracy of the event times using SVHM compared to existing machine learning methods and standard conventional approaches. Finally, we analyze two real world biomedical study data where we use clinical markers and neuroimaging biomarkers to predict age-at- onset of a disease, and demonstrate superiority of SVHM in distinguishing high risk versus low risk subjects.

JMLR Other · 发表于 2016

Subspace Learning with Partial Information

Alon Gonen, Dan Rosenbaum, Yonina C. Eldar, Shai Shalev-Shwartz

摘要

The goal of subspace learning is to find a $k$-dimensional subspace of $\mathbb{R}^d$, such that the expected squared distance between instance vectors and the subspace is as small as possible. In this paper we study subspace learning in a partial information setting, in which the learner can only observe $r \le d$ attributes from each instance vector. We propose several efficient algorithms for this task, and analyze their sample complexity.

JMLR Diagnostics & Prediction High-Dimensional · 发表于 2016

Structure-Leveraged Methods in Breast Cancer Risk Prediction

Jun Fan, Yirong Wu, Ming Yuan, David Page, Jie Liu, Irene M. Ong, Peggy Peissig, Elizabeth Burnside

摘要

Predicting breast cancer risk has long been a goal of medical research in the pursuit of precision medicine. The goal of this study is to develop novel penalized methods to improve breast cancer risk prediction by leveraging structure information in electronic health records. We conducted a retrospective case- control study, garnering 49 mammography descriptors and 77 high- frequency/low-penetrance single-nucleotide polymorphisms (SNPs) from an existing personalized medicine data repository. Structured mammography reports and breast imaging features have long been part of a standard electronic health record (EHR), and genetic markers likely will be in the near future. Lasso and its variants are widely used approaches to integrated learning and feature selection, and our methodological contribution is to incorporate the dependence structure among the features into these approaches. More specifically, we propose a new methodology by combining group penalty and $\ell^p$ ($1\leq p\leq2$) fusion penalty to improve breast cancer risk prediction, taking into account structure information in mammography descriptors and SNPs. We demonstrate that our method provides benefits that are both statistically significant and potentially significant to people's lives.

JMLR Bayesian Methods Networks & Graphs · 发表于 2016

Structure Learning in Bayesian Networks of a Moderate Size by Efficient Sampling

Ru He, Jin Tian, Huaiqing Wu

摘要

We study the Bayesian model averaging approach to learning Bayesian network structures (DAGs) from data. We develop new algorithms including the first algorithm that is able to efficiently sample DAGs of a moderate size (with up to about 25 variables) according to the exact structure posterior. The DAG samples can then be used to construct estimators for the posterior of any feature. We theoretically prove good properties of our estimators and empirically show that our estimators considerably outperform the estimators from the previous state- of-the-art methods.

JMLR Bayesian Methods Networks & Graphs · 发表于 2016

Structure Discovery in Bayesian Networks by Sampling Partial Orders

Teppo Niinimäki, Pekka Parviainen, Mikko Koivisto

摘要

We present methods based on Metropolis-coupled Markov chain Monte Carlo (MC3) and annealed importance sampling (AIS) for estimating the posterior distribution of Bayesian networks. The methods draw samples from an appropriate distribution of partial orders on the nodes, continued by sampling directed acyclic graphs (DAGs) conditionally on the sampled partial orders. We show that the computations needed for the sampling algorithms are feasible as long as the encountered partial orders have relatively few down-sets. While the algorithms assume suitable modularity properties of the priors, arbitrary priors can be handled by dividing the importance weight of each sampled DAG by the number of topological sorts it has---we give a practical dynamic programming algorithm to compute these numbers. Our empirical results demonstrate that the presented partial-order- based samplers are superior to previous Markov chain Monte Carlo methods, which sample DAGs either directly or via linear orders on the nodes. The results also suggest that the convergence rate of the estimators based on AIS are competitive to those of MC3. Thus AIS is the preferred method, as it enables easier large- scale parallelization and, in addition, supplies good probabilistic lower bound guarantees for the marginal likelihood of the model.

JMLR Other · 发表于 2016

StructED: Risk Minimization in Structured Prediction

Yossi Adi, Joseph Keshet

摘要

Structured tasks are distinctive: each task has its own measure of performance, such as the word error rate in speech recognition, the BLEU score in machine translation, the NDCG score in information retrieval, or the intersection-over-union score in visual object segmentation. This paper presents StructED, a software package for learning structured prediction models with training methods that aimed at optimizing the task measure of performance. The package was written in Java and released under the MIT license. It can be downloaded from adiyoss.github.io/StructED.

JMLR Bayesian Methods Meta-Analysis & Data Integration · 发表于 2016

String and Membrane Gaussian Processes

Yves-Laurent Kom Samo, Stephen J. Roberts

摘要

In this paper we introduce a novel framework for making exact nonparametric Bayesian inference on latent functions that is particularly suitable for Big Data tasks. Firstly, we introduce a class of stochastic processes we refer to as string Gaussian processes (string GPs which are not to be mistaken for Gaussian processes operating on text). We construct string GPs so that their finite- dimensional marginals exhibit suitable local conditional independence structures, which allow for scalable, distributed, and flexible nonparametric Bayesian inference, without resorting to approximations, and while ensuring some mild global regularity constraints. Furthermore, string GP priors naturally cope with heterogeneous input data, and the gradient of the learned latent function is readily available for explanatory analysis. Secondly, we provide some theoretical results relating our approach to the standard GP paradigm. In particular, we prove that some string GPs are Gaussian processes, which provides a complementary global perspective on our framework. Finally, we derive a scalable and distributed MCMC scheme for supervised learning tasks under string GP priors. The proposed MCMC scheme has computational time complexity $\mathcal{O}(N)$ and memory requirement $\mathcal{O}(dN)$, where $N$ is the data size and $d$ the dimension of the input space. We illustrate the efficacy of the proposed approach on several synthetic and real-world data sets, including a data set with $6$ millions input points and $8$ attributes.

JMLR Networks & Graphs Machine Learning · 发表于 2016

Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches

Jure Žbontar, Yann LeCun

摘要

We present a method for extracting depth information from a rectified image pair. Our approach focuses on the first stage of many stereo algorithms: the matching cost computation. We approach the problem by learning a similarity measure on small image patches using a convolutional neural network. Training is carried out in a supervised manner by constructing a binary classification data set with examples of similar and dissimilar pairs of patches. We examine two network architectures for this task: one tuned for speed, the other for accuracy. The output of the convolutional neural network is used to initialize the stereo matching cost. A series of post-processing steps follow: cross-based cost aggregation, semiglobal matching, a left-right consistency check, subpixel enhancement, a median filter, and a bilateral filter. We evaluate our method on the KITTI 2012, KITTI 2015, and Middlebury stereo data sets and show that it outperforms other approaches on all three data sets.

JMLR Networks & Graphs Latent Variables & Mixtures · 发表于 2016

Statistical-Computational Tradeoffs in Planted Problems and Submatrix Localization with a Growing Number of Clusters and Submatrices

Yudong Chen, Jiaming Xu

摘要

We consider two closely related problems: planted clustering and submatrix localization. In the planted clustering problem, a random graph is generated based on an underlying cluster structure of the nodes; the task is to recover these clusters given the graph. The submatrix localization problem concerns locating hidden submatrices with elevated means inside a large real-valued random matrix. Of particular interest is the setting where the number of clusters/submatrices is allowed to grow unbounded with the problem size. These formulations cover several classical models such as planted clique, planted densest subgraph, planted partition, planted coloring, and the stochastic block model, which are widely used for studying community detection, graph clustering and bi-clustering.

JMLR Networks & Graphs · 发表于 2016

Stable Graphical Models

Navodit Misra, Ercan E. Kuruoglu

摘要

Stable random variables are motivated by the central limit theorem for densities with (potentially) unbounded variance and can be thought of as natural generalizations of the Gaussian distribution to skewed and heavy-tailed phenomenon. In this paper, we introduce $\alpha$-stable graphical ($\alpha$-SG) models, a class of multivariate stable densities that can also be represented as Bayesian networks whose edges encode linear dependencies between random variables. One major hurdle to the extensive use of stable distributions is the lack of a closed- form analytical expression for their densities. This makes penalized maximum-likelihood based learning computationally demanding. We establish theoretically that the Bayesian information criterion (BIC) can asymptotically be reduced to the computationally more tractable minimum dispersion criterion (MDC) and develop StabLe, a structure learning algorithm based on MDC. We use simulated datasets for five benchmark network topologies to empirically demonstrate how StabLe improves upon ordinary least squares (OLS) regression. We also apply StabLe to microarray gene expression data for lymphoblastoid cells from 727 individuals belonging to eight global population groups. We establish that StabLe improves test set performance relative to OLS via ten-fold cross-validation. Finally, we develop SGEX, a method for quantifying differential expression of genes between different population groups.

JMLR Other · 发表于 2016

Stability and Generalization in Structured Prediction

Ben London, Bert Huang, Lise Getoor

摘要

Structured prediction models have been found to learn effectively from a few large examples--- sometimes even just one. Despite empirical evidence, canonical learning theory cannot guarantee generalization in this setting because the error bounds decrease as a function of the number of examples. We therefore propose new PAC-Bayesian generalization bounds for structured prediction that decrease as a function of both the number of examples and the size of each example. Our analysis hinges on the stability of joint inference and the smoothness of the data distribution. We apply our bounds to several common learning scenarios, including max-margin and soft-max training of Markov random fields. Under certain conditions, the resulting error bounds can be far more optimistic than previous results and can even guarantee generalization from a single large example.

JMLR Other · 发表于 2016

Spectral Ranking using Seriation

Fajwel Fogel, Alexandre d'Aspremont, Milan Vojnovic

摘要

We describe a seriation algorithm for ranking a set of items given pairwise comparisons between these items. Intuitively, the algorithm assigns similar rankings to items that compare similarly with all others. It does so by constructing a similarity matrix from pairwise comparisons, using seriation methods to reorder this matrix and construct a ranking. We first show that this spectral seriation algorithm recovers the true ranking when all pairwise comparisons are observed and consistent with a total order. We then show that ranking reconstruction is still exact when some pairwise comparisons are corrupted or missing, and that seriation based spectral ranking is more robust to noise than classical scoring methods. Finally, we bound the ranking error when only a random subset of the comparions are observed. An additional benefit of the seriation formulation is that it allows us to solve semi-supervised ranking problems. Experiments on both synthetic and real datasets demonstrate that seriation based spectral ranking achieves competitive and in some cases superior performance compared to classical ranking methods.

JMLR Other · 发表于 2016

Spectral Methods Meet EM: A Provably Optimal Algorithm for Crowdsourcing

Yuchen Zhang, Xi Chen, Dengyong Zhou, Michael I. Jordan

摘要

Crowdsourcing is a popular paradigm for effectively collecting labels at low cost. The Dawid-Skene estimator has been widely used for inferring the true labels from the noisy labels provided by non-expert crowdsourcing workers. However, since the estimator maximizes a non-convex log-likelihood function, it is hard to theoretically justify its performance. In this paper, we propose a two-stage efficient algorithm for multi-class crowd labeling problems. The first stage uses the spectral method to obtain an initial estimate of parameters. Then the second stage refines the estimation by optimizing the objective function of the Dawid-Skene estimator via the EM algorithm. We show that our algorithm achieves the optimal convergence rate up to a logarithmic factor. We conduct extensive experiments on synthetic and real datasets. Experimental results demonstrate that the proposed algorithm is comparable to the most accurate empirical approach, while outperforming several other recently proposed methods.

JMLR High-Dimensional Nonparametric & Semiparametric · 发表于 2016

Sparsity and Error Analysis of Empirical Feature-Based Regularization Schemes

Xin Guo, Jun Fan, Ding-Xuan Zhou

摘要

We consider a learning algorithm generated by a regularization scheme with a concave regularizer for the purpose of achieving sparsity and good learning rates in a least squares regression setting. The regularization is induced for linear combinations of empirical features, constructed in the literatures of kernel principal component analysis and kernel projection machines, based on kernels and samples. In addition to the separability of the involved optimization problem caused by the empirical features, we carry out sparsity and error analysis, giving bounds in the norm of the reproducing kernel Hilbert space, based on a priori conditions which do not require assumptions on sparsity in terms of any basis or system. In particular, we show that as the concave exponent $q$ of the concave regularizer increases to $1$, the learning ability of the algorithm improves. Some numerical simulations for both artificial and real MHC-peptide binding data involving the $\ell^q$ regularizer and the SCAD penalty are presented to demonstrate the sparsity and error analysis.

JMLR High-Dimensional · 发表于 2016

Sparse PCA via Covariance Thresholding

Yash Deshpande, Andrea Montanari

摘要

In sparse principal component analysis we are given noisy observations of a low-rank matrix of dimension $n\times p$ and seek to reconstruct it under additional sparsity assumptions. In particular, we assume here each of the principal components $v_1,\dots,v_r$ has at most $s_0$ non-zero entries. We are particularly interested in the high dimensional regime wherein $p$ is comparable to, or even much larger than $n$.

JMLR Other · 发表于 2016

Should We Really Use Post-Hoc Tests Based on Mean-Ranks?

Alessio Benavoli, Giorgio Corani, Francesca Mangili

摘要

The statistical comparison of multiple algorithms over multiple data sets is fundamental in machine learning. This is typically carried out by the Friedman test. When the Friedman test rejects the null hypothesis, multiple comparisons are carried out to establish which are the significant differences among algorithms. The multiple comparisons are usually performed using the mean-ranks test. The aim of this technical note is to discuss the inconsistencies of the mean-ranks post-hoc test with the goal of discouraging its use in machine learning as well as in medicine, psychology, etc.. We show that the outcome of the mean-ranks test depends on the pool of algorithms originally included in the experiment. In other words, the outcome of the comparison between algorithms $A$ and $B$ depends also on the performance of the other algorithms included in the original experiment. This can lead to paradoxical situations. For instance the difference between $A$ and $B$ could be declared significant if the pool comprises algorithms $C,D,E$ and not significant if the pool comprises algorithms $F,G,H$. To overcome these issues, we suggest instead to perform the multiple comparison using a test whose outcome only depends on the two algorithms being compared, such as the sign-test or the Wilcoxon signed-rank test.

Econometrica Econometrics & Economics · 发表于 2016

Shadow Insurance

Ralph S. J. Koijen, Motohiro Yogo

摘要

Life insurers use reinsurance to move liabilities from regulated and rated companies that sell policies to shadow reinsurers, which are less regulated and unrated off-balance-sheet entities within the same insurance group. U.S. life insurance and annuity liabilities ceded to shadow reinsurers grew from $11 billion in 2002 to $364 billion in 2012. Life insurers using shadow insurance, which capture half of the market share, ceded 25 cents of every dollar insured to shadow reinsurers in 2012, up from 2 cents in 2002. By relaxing capital requirements, shadow insurance could reduce the marginal cost of issuing policies and thereby improve retail market efficiency. However, shadow insurance could also reduce risk-based capital and increase expected loss for the industry. We model and quantify these effects based on publicly available data and plausible assumptions. *(abstract via OpenAlex)*

JMLR Bayesian Methods Nonparametric & Semiparametric · 发表于 2016

Semiparametric Mean Field Variational Bayes: General Principles and Numerical Issues

David Rohde, Matt P. Wand

摘要

We introduce the term semiparametric mean field variational Bayes to describe the relaxation of mean field variational Bayes in which some density functions in the product density restriction are pre-specified to be members of convenient parametric families. This notion has appeared in various guises in the mean field variational Bayes literature during its history and we endeavor to unify this important topic. We lay down a general framework and explain how previous relevant methodologies fall within this framework. A major contribution is elucidation of numerical issues that impact semiparametric mean field variational Bayes in practice.

Econometrica Econometrics & Economics · 发表于 2016

Search With Adverse Selection

Stephan Lauermann, Asher Wolinsky

摘要

This paper analyzes a search model with asymmetric information of the common values variety. The basic features of this environment resemble those of a common values (procurement) auction, except that the searcher in our model, who is the counterpart of the auctioneer in the auction model, encounters trading partners through costly sequential search. The main objective is to understand how the combination of search activity and information asymmetry affects prices and welfare. We specifically inquire about the extent of information aggregation by the price –how close the equilibrium prices are to the full information prices–when the search frictions are small. Roughly speaking, we conclude that information is aggregated less well in the search environment than it is in the corresponding auction environment. We trace this to a stronger form of winner’s curse that is present in the search scenario. This understanding is a central qualitative insight of this paper, which is likely to have implications beyond the narrow confines of our model. We also look at the efficiency perspective and examine the relations between total surplus and the informativenss of the signal technology available to the uninformed. We conclude that total surplus is not monotone in the quality of the signals. *(abstract via OpenAlex)*

JMLR Optimization & Learning Theory Latent Variables & Mixtures · 发表于 2016

Scaling-up Empirical Risk Minimization: Optimization of Incomplete $U$-statistics

Stephan Clémençon, Igor Colin, Aurélien Bellet

摘要

In a wide range of statistical learning problems such as ranking, clustering or metric learning among others, the risk is accurately estimated by $U$-statistics of degree $d\geq 1$, i.e. functionals of the training data with low variance that take the form of averages over $k$-tuples. From a computational perspective, the calculation of such statistics is highly expensive even for a moderate sample size $n$, as it requires averaging $O(n^d)$ terms. This makes learning procedures relying on the optimization of such data functionals hardly feasible in practice. It is the major goal of this paper to show that, strikingly, such empirical risks can be replaced by drastically computationally simpler Monte-Carlo estimates based on $O(n)$ terms only, usually referred to as incomplete $U$-statistics, without damaging the $O_{\mathbb{P}}(1/\sqrt{n})$ learning rate of Empirical Risk Minimization (ERM) procedures. For this purpose, we establish uniform deviation results describing the error made when approximating a $U$-process by its incomplete version under appropriate complexity assumptions. Extensions to model selection, fast rate situations and various sampling techniques are also considered, as well as an application to stochastic gradient descent for ERM. Finally, numerical examples are displayed in order to provide strong empirical evidence that the approach we promote largely surpasses more naive subsampling techniques.

JMLR Bayesian Methods Networks & Graphs · 发表于 2016

Scalable Learning of Bayesian Network Classifiers

Ana M. Martínez, Geoffrey I. Webb, Shenglei Chen, Nayyar A. Zaidi

摘要

Ever increasing data quantity makes ever more urgent the need for highly scalable learners that have good classification performance. Therefore, an out-of-core learner with excellent time and space complexity, along with high expressivity (that is, capacity to learn very complex multivariate probability distributions) is extremely desirable. This paper presents such a learner. We propose an extension to the $k$-dependence Bayesian classifier (KDB) that discriminatively selects a sub- model of a full KDB classifier. It requires only one additional pass through the training data, making it a three-pass learner. Our extensive experimental evaluation on $16$ large data sets reveals that this out-of-core algorithm achieves competitive classification performance, and substantially better training and classification time than state-of-the-art in-core learners such as random forest and linear and non-linear logistic regression.

JMLR Bayesian Methods Survey & Sampling · 发表于 2016

Scalable Approximate Bayesian Inference for Outlier Detection under Informative Sampling

Terrance D. Savitsky

摘要

Government surveys of business establishments receive a large volume of submissions where a small subset contain errors. Analysts need a fast-computing algorithm to flag this subset due to a short time window between collection and reporting. We offer a computationally-scalable optimization method based on non-parametric mixtures of hierarchical Dirichlet processes that allows discovery of multiple industry-indexed local partitions linked to a set of global cluster centers. Outliers are nominated as those clusters containing few observations. We extend an existing approach with a new merge step that reduces sensitivity to hyperparameter settings. Survey data are typically acquired under an informative sampling design where the probability of inclusion depends on the surveyed response such that the distribution for the observed sample is different from the population. We extend the derivation of a penalized objective function to use a pseudo-posterior that incorporates sampling weights that undo the informative design. We provide a simulation study to demonstrate that our approach produces unbiased estimation for the outlying cluster under informative sampling. The method is applied for outlier nomination for the Current Employment Statistics survey conducted by the Bureau of Labor Statistics.

JMLR Other · 发表于 2016

SPSD Matrix Approximation vis Column Selection: Theories, Algorithms, and Extensions

Shusen Wang, Luo Luo, Zhihua Zhang

摘要

Symmetric positive semidefinite (SPSD) matrix approximation is an important problem with applications in kernel methods. However, existing SPSD matrix approximation methods such as the Nyström method only have weak error bounds. In this paper we conduct in-depth studies of an SPSD matrix approximation model and establish strong relative-error bounds. We call it the prototype model for it has more efficient and effective extensions, and some of its extensions have high scalability. Though the prototype model itself is not suitable for large- scale data, it is still useful to study its properties, on which the analysis of its extensions relies. This paper offers novel theoretical analysis, efficient algorithms, and a highly accurate extension. First, we establish a lower error bound for the prototype model and improve the error bound of an existing column selection algorithm to match the lower bound. In this way, we obtain the first optimal column selection algorithm for the prototype model. We also prove that the prototype model is exact under certain conditions. Second, we develop a simple column selection algorithm with a provable error bound. Third, we propose a so-called spectral shifting model to make the approximation more accurate when the eigenvalues of the matrix decay slowly, and the improvement is theoretically quantified. The spectral shifting method can also be applied to improve other SPSD matrix approximation models.

JMLR Other · 发表于 2016

Rounding-based Moves for Semi-Metric Labeling

M. Pawan Kumar, Puneet K. Dokania

摘要

Semi-metric labeling is a special case of energy minimization for pairwise Markov random fields. The energy function consists of arbitrary unary potentials, and pairwise potentials that are proportional to a given semi-metric distance function over the label set. Popular methods for solving semi-metric labeling include (i) move-making algorithms, which iteratively solve a minimum $st$-cut problem; and (ii) the linear programming ( LP) relaxation based approach. In order to convert the fractional solution of the LP relaxation to an integer solution, several randomized rounding procedures have been developed in the literature. We consider a large class of parallel rounding procedures, and design move-making algorithms that closely mimic them. We prove that the multiplicative bound of a move-making algorithm exactly matches the approximation factor of the corresponding rounding procedure for any arbitrary distance function. Our analysis includes all known results for move- making algorithms as special cases.

Econometrica Other · 发表于 2016

Robust Contracts in Continuous Time

Jianjun Miao, Alejandro Rivera

摘要

We study two types of robust contracting problem under hidden action in continuous time. In type I problem, the principal is ambiguous about the project cash flows, while he is ambiguous about the agent’s beliefs in type II problem. The principal designs a robust contract that maximizes his utility under the worst-case scenario subject to the agent’s incentive and participation constraints. We implement the optimal contract by cash reserves, debt and equity. In addition to receiving ordinary dividends when cash reserves reach a threshold, outside equity holders also receive special dividends or inject cash in the cash reserves to hedge against model uncertainty. Ambiguity aversion lowers outside securities value and raises the credit yield spread. It generates equity premium for type I problem, but not for type II problem. The equity premium and the credit yield spread are state dependent and high for distressed firms with low cash reserves. *(abstract via OpenAlex)*

Econometrica Other · 发表于 2016

Robust Confidence Regions for Incomplete Models

Larry G. Epstein, Hiroaki Kaido, Kyoungwon Seo

摘要

Call an economic model incomplete if it does not generate a probabilistic prediction even given knowledge of all parameter values.We propose a method of inference about unknown parameters for such models that is robust to heterogeneity and dependence of unknown form.The key is a Central Limit Theorem for belief functions; robust con…dence regions are then constructed in a fashion paralleling the classical approach.Monte Carlo simulations support tractability of the method and demonstrate its enhanced robustness relative to existing methods. *(abstract via OpenAlex)*

JMLR Machine Learning · 发表于 2016

Revisiting the Nyström Method for Improved Large-scale Machine Learning

Alex Gittens, Michael W. Mahoney

摘要

We reconsider randomized algorithms for the low-rank approximation of symmetric positive semi-definite (SPSD) matrices such as Laplacian and kernel matrices that arise in data analysis and machine learning applications. Our main results consist of an empirical evaluation of the performance quality and running time of sampling and projection methods on a diverse suite of SPSD matrices. Our results highlight complementary aspects of sampling versus projection methods; they characterize the effects of common data preprocessing steps on the performance of these algorithms; and they point to important differences between uniform sampling and nonuniform sampling methods based on leverage scores. In addition, our empirical results illustrate that existing theory is so weak that it does not provide even a qualitative guide to practice. Thus, we complement our empirical results with a suite of worst- case theoretical bounds for both random sampling and random projection methods. These bounds are qualitatively superior to existing bounds---e.g., improved additive-error bounds for spectral and Frobenius norm error and relative-error bounds for trace norm error---and they point to future directions to make these algorithms useful in even larger-scale machine learning applications.

Econometrica Other · 发表于 2016

Reputational Bargaining and Deadlines

Jack Fanning

摘要

I highlight how reputational concerns provide a natural explanation for “deadline effects,” the high frequency of deals prior to a deadline in bargaining. Rational agents imitate the demands of obstinate behavioral types and engage in brinkmanship in the face of uncertainty about the deadline's arrival. I also identify how surplus is divided when the prior probability of behavioral types is vanishingly small. If behavioral types are committed to fixed demands, outcomes converge to the Nash bargaining solution regardless of agents' respective impatience. If behavioral types can adopt more complex demand strategies, outcomes converge to the solution of an alternating offers game without behavioral types for the deadline environment. *(abstract via OpenAlex)*

Econometrica Econometrics & Economics · 发表于 2016

Relational Incentive Contracts With Persistent Private Information

James M. Malcomson

摘要

This paper investigates relational incentive contracts with continuous, privately-observed agent types that are persistent over time. With fixed agent types, full separation is not possible when continuation equilibrium payoffs following revelation are on the Pareto frontier of attainable payoffs. This result is related to the ratchet effect in that: (1) a type imitating a less productive type receives an information rent and (2) with full separation, one imitating a more productive type receives the same future payoff as that more productive type. However, the reason for (2) is fundamentally different than with the ratchet effect. It arises from the dynamic enforcement requirement in relational contracts, not from the principal having all the bargaining power, and applies whatever the distribution between principal and agent of the future gains from the relationship (that is, whatever the point on the Pareto frontier). This result extends to sufficiently persistent types under certain conditions. *(abstract via OpenAlex)*

JMLR Nonparametric & Semiparametric · 发表于 2016

Regularized Policy Iteration with Nonparametric Function Spaces

Amir-massoud Farahmand, Mohammad Ghavamzadeh, Csaba Szepesvári, Shie Mannor

摘要

We study two regularization-based approximate policy iteration algorithms, namely REG-LSPI and REG-BRM, to solve reinforcement learning and planning problems in discounted Markov Decision Processes with large state and finite action spaces. The core of these algorithms are the regularized extensions of the Least- Squares Temporal Difference (LSTD) learning and Bellman Residual Minimization (BRM), which are used in the algorithms' policy evaluation steps. Regularization provides a convenient way to control the complexity of the function space to which the estimated value function belongs and as a result enables us to work with rich nonparametric function spaces. We derive efficient implementations of our methods when the function space is a reproducing kernel Hilbert space. We analyze the statistical properties of REG-LSPI and provide an upper bound on the policy evaluation error and the performance loss of the policy returned by this method. Our bound shows the dependence of the loss on the number of samples, the capacity of the function space, and some intrinsic properties of the underlying Markov Decision Process. The dependence of the policy evaluation bound on the number of samples is minimax optimal. This is the first work that provides such a strong guarantee for a nonparametric approximate policy iteration algorithm. (This work is an extension of the NIPS 2008 conference paper by Farahmand et al. (2009b).)

JMLR Other · 发表于 2016

Refined Error Bounds for Several Learning Algorithms

Steve Hanneke

摘要

This article studies the achievable guarantees on the error rates of certain learning algorithms, with particular focus on refining logarithmic factors. Many of the results are based on a general technique for obtaining bounds on the error rates of sample-consistent classifiers with monotonic error regions, in the realizable case. We prove bounds of this type expressed in terms of either the VC dimension or the sample compression size. This general technique also enables us to derive several new bounds on the error rates of general sample-consistent learning algorithms, as well as refined bounds on the label complexity of the CAL active learning algorithm. Additionally, we establish a simple necessary and sufficient condition for the existence of a distribution-free bound on the error rates of all sample- consistent learning rules, converging at a rate inversely proportional to the sample size. We also study learning in the presence of classification noise, deriving a new excess error rate guarantee for general VC classes under Tsybakov's noise condition, and establishing a simple and general necessary and sufficient condition for the minimax excess risk under bounded noise to converge at a rate inversely proportional to the sample size.

JMLR High-Dimensional · 发表于 2016

Rate Optimal Denoising of Simultaneously Sparse and Low Rank Matrices

Dan Yang, Zongming Ma, Andreas Buja

摘要

We study minimax rates for denoising simultaneously sparse and low rank matrices in high dimensions. We show that an iterative thresholding algorithm achieves (near) optimal rates adaptively under mild conditions for a large class of loss functions. Numerical experiments on synthetic datasets also demonstrate the competitive performance of the proposed method.

SiM Other · 发表于 2016

Random effects meta‐analysis: Coverage performance of 95 % confidence and prediction intervals following REML estimation

Christopher Partlett, Richard D. Riley

摘要

A random effects meta‐analysis combines the results of several independent studies to summarise the evidence about a particular measure of interest, such as a treatment effect. The approach allows for unexplained between‐study heterogeneity in the true treatment effect by incorporating random study effects about the overall mean. The variance of the mean effect estimate is conventionally calculated by assuming that the between study variance is known; however, it has been demonstrated that this approach may be inappropriate, especially when there are few studies. Alternative methods that aim to account for this uncertainty, such as Hartung–Knapp, Sidik–Jonkman and Kenward–Roger, have been proposed and shown to improve upon the conventional approach in some situations. In this paper, we use a simulation study to examine the performance of several of these methods in terms of the coverage of the 95 % confidence and prediction intervals derived from a random effects meta‐analysis estimated using restricted maximum likelihood. We show that, in terms of the confidence intervals, the Hartung–Knapp correction performs well across a wide‐range of scenarios and outperforms other methods when heterogeneity was large and/or study sizes were similar. However, the coverage of the Hartung–Knapp method is slightly too low when the heterogeneity is low ( I 2 30 % ) and study sizes are similar. In other situations, especially when heterogeneity is small and the study sizes are quite varied, the coverage is far too low and could not be consistently improved by either increasing the number of studies, altering the degrees of freedom or using variance inflation methods. Therefore, researchers should be cautious in deriving 95 % prediction intervals following a frequentist random‐effects meta‐analysis until a more reliable solution is identified. © 2016 The Authors. Statistics in Medicine

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 页