研究园简易版 · 经典书籍 · 期刊文章 · 有原文 · 完整版(可多选筛选、搜索) · 首页
2024 年的期刊文章 · 第 26 页
共 1515 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2024-01-11 到 2024。
按期刊、按主题看 →
JASA Testing & Uncertainty · 发表于 2024-01-11
Yang Cao, Xinwei Sun, Yuan Yao
摘要
Multiple comparisons in hypothesis testing often encounter structural constraints in various applications. For instance, in structural Magnetic Resonance Imaging for Alzheimer’s Disease, the focus extends beyond examining atrophic brain regions to include comparisons of anatomically adjacent regions. These constraints can be modeled as linear transformations of parameters, where the sign patterns play a crucial role in estimating directional effects. This class of problems, encompassing total variations, wavelet transforms, fused LASSO, trend filtering, and more, presents an open challenge in effectively controlling the directional false discovery rate. In this article, we propose an extended Split Knockoff method specifically designed to address the control of directional false discovery rate under linear transformations. Our proposed approach relaxes the stringent linear manifold constraint to its neighborhood, employing a variable splitting technique commonly used in optimization. This methodology yields an orthogonal design that benefits both power and directional false discovery rate control. By incorporating a sample splitting scheme, we achieve effective control of the directional false discovery rate, with a notable reduction to zero as the relaxed neighborhood expands. To demonstrate the efficacy of our method, we conduct simulation experiments and apply it to two real-world scenarios: Alzheimer’s Disease analysis and human age comparisons. Supplementary materials for this article are available online. *(abstract via OpenAlex)*
SiM Causal Inference Clinical Trials Bayesian Methods · 发表于 2024-01-11
Ian C. Marschner
摘要
An attractive feature of using a Bayesian analysis for a clinical trial is that knowledge and uncertainty about the treatment effect is summarized in a posterior probability distribution. Researchers often find probability statements about treatment effects highly intuitive and the fact that this is not accommodated in frequentist inference is a disadvantage. At the same time, the requirement to specify a prior distribution in order to obtain a posterior distribution is sometimes an artificial process that may introduce subjectivity or complexity into the analysis. This paper considers a compromise involving confidence distributions, which are probability distributions that summarize uncertainty about the treatment effect without the need for a prior distribution and in a way that is fully compatible with frequentist inference. The concept of a confidence distribution provides a posterior–like probability distribution that is distinct from, but exists in tandem with, the relative frequency interpretation of probability used in frequentist inference. Although they have been discussed for decades, confidence distributions are not well known among clinical trial statisticians and the goal of this paper is to discuss their use in analyzing treatment effects from randomized trials. As well as providing an introduction to confidence distributions, some illustrative examples relevant to clinical trials are presented, along with various case studies based on real clinical trials. It is recommended that trial statisticians consider presenting confidence distributions for treatment effects when reporting analyses of clinical trials.
SiM Longitudinal Data Networks & Graphs Latent Variables & Mixtures · 发表于 2024-01-10
Andrea Cremaschi, Maria De Iorio, Narasimhan Kothandaraman, Fabian Yap, Mya Thway Tint, Johan Eriksson
摘要
The prevalence of chronic non‐communicable diseases such as obesity has noticeably increased in the last decade. The study of these diseases in early life is of paramount importance in determining their course in adult life and in supporting clinical interventions. Recently, attention has been drawn to approaches that study the alteration of metabolic pathways in obese children. In this work, we propose a novel joint modeling approach for the analysis of growth biomarkers and metabolite associations, to unveil metabolic pathways related to childhood obesity. Within a Bayesian framework, we flexibly model the temporal evolution of growth trajectories and metabolic associations through the specification of a joint nonparametric random effect distribution, with the main goal of clustering subjects, thus identifying risk sub‐groups. Growth profiles as well as patterns of metabolic associations determine the clustering structure. Inclusion of risk factors is straightforward through the specification of a regression term. We demonstrate the proposed approach on data from the Growing Up in Singapore Towards healthy Outcomes cohort study, based in Singapore. Posterior inference is obtained via a tailored MCMC algorithm, involving a nonparametric prior with mixed support. Our analysis has identified potential key pathways in obese children that allow for the exploration of possible molecular mechanisms associated with childhood obesity.
JASA Clinical Trials Causal Inference Machine Learning · 发表于 2024-01-09
Bingkai Wang, Chan Park, Dylan S. Small, Fan Li
摘要
Cluster-randomized experiments are increasingly used to evaluate interventions in routine practice conditions, and researchers often adopt model-based methods with covariate adjustment in the statistical analyses. However, the validity of model-based covariate adjustment remains unclear when the working models are misspecified, leading to ambiguity of estimands and risk of bias. In this article, we first adapt two model-based methods-generalized estimating equations and linear mixed models-with weighted g-computation to achieve robust inference for cluster-average and individual-average treatment effects. To further overcome the limitations of model-based covariate adjustment methods, we propose efficient estimators for each estimand that allow for flexible covariate adjustment and additionally address cluster size variation dependent on treatment assignment and other cluster characteristics. Such cluster size variations often occur post-randomization and, if ignored, can lead to bias of model-based estimators. For our proposed covariate-adjusted estimators, we prove that when the nuisance functions are consistently estimated by machine learning algorithms, the estimators are consistent, asymptotically normal, and efficient. When the nuisance functions are estimated via parametric working models, the estimators are triply-robust. Simulation studies and analyses of three real-world cluster-randomized experiments demonstrate that the proposed methods are superior to existing alternatives. *(abstract via PubMed)*
SMMR Survival Analysis Functional Data Diagnostics & Prediction · 发表于 2024-01-09
Daniel Gomon, Hein Putter, Marta Fiocco, Mirko Signorelli
摘要
Dynamically predicting patient survival probabilities using longitudinal measurements has become of great importance with routine data collection becoming more common. Many existing models utilize a multi-step landmarking approach for this problem, mostly due to its ease of use and versatility but unfortunately most fail to do so appropriately. In this article we make use of multivariate functional principal component analysis to summarize the available longitudinal information, and employ a Cox proportional hazards model for prediction. Additionally, we consider a centred functional principal component analysis procedure in an attempt to remove the natural variation incurred by the difference in age of the considered subjects. We formalize the difference between a ‘relaxed’ landmarking approach where only validation data is landmarked and a ‘strict’ landmarking approach where both the training and validation data are landmarked. We show that a relaxed landmarking approach fails to effectively use the information contained in the longitudinal outcomes, thereby producing substantially worse prediction accuracy than a strict landmarking approach.
JASA Testing & Uncertainty · 发表于 2024-01-09
Giuseppe Cavaliere, Sílvia Gonçalves, Morten Ørregaard Nielsen, Edoardo Zanelli
摘要
We consider bootstrap inference for estimators which are (asymptotically) biased. We show that, even when the bias term cannot be consistently estimated, valid inference can be obtained by proper implementations of the bootstrap. Specifically, we show that the prepivoting approach of Beran (1987, 1988), originally proposed to deliver higher-order refinements, restores bootstrap validity by transforming the original bootstrap p-value into an asymptotically uniform random variable. We propose two different implementations of prepivoting (plug-in and double bootstrap), and provide general high-level conditions that imply validity of bootstrap inference. To illustrate the practical relevance and implementation of our results, we discuss five examples: (i) inference on a target parameter based on model averaging; (ii) ridge-type regularized estimators; (iii) nonparametric regression; (iv) a location model for infinite variance data; and (v) dynamic panel data models. *(abstract via OpenAlex)*
JASA Other · 发表于 2024-01-09
Christopher Harshaw, Fredrik Sävje, Daniel A. Spielman, Peng Zhang
摘要
The design of experiments involves a compromise between covariate balance and robustness.This article provides a formalization of this tradeoff and describes an experimental design that allows experimenters to navigate it.The design is specified by a robustness parameter that bounds the worst-case mean squared error of an estimator of the average treatment effect.Subject to the experimenter's desired level of robustness, the design aims to simultaneously balance all linear functions of potentially many covariates.Less robustness allows for more balance.We show that the mean squared error of the estimator is bounded in finite samples by the minimum of the loss function of an implicit ridge regression of the potential outcomes on the covariates.Asymptotically, the design perfectly balances all linear functions of a growing number of covariates with a diminishing reduction in robustness, effectively allowing experimenters to escape the compromise between balance and robustness in large samples.Finally, we describe conditions that ensure asymptotic normality and provide a conservative variance estimator, which facilitate the construction of asymptotically valid confidence intervals.Supplementary materials for this article are available online. *(abstract via OpenAlex)*
JASA Bayesian Methods Spatial & Time Series High-Dimensional · 发表于 2024-01-08
Anna Menacher, Thomas E. Nichols, Chris Holmes, Habib Ganjgahi
摘要
Neural demyelination and brain damage accumulated in white matter appear as hyperintense areas on T2-weighted MRI scans in the form of lesions. Modeling binary images at the population level, where each voxel represents the existence of a lesion, plays an important role in understanding aging and inflammatory diseases. We propose a scalable hierarchical Bayesian spatial model, called BLESS, capable of handling binary responses by placing continuous spike-and-slab mixture priors on spatially-varying parameters and enforcing spatial dependency on the parameter dictating the amount of sparsity within the probability of inclusion. The use of mean-field variational inference with dynamic posterior exploration, which is an annealing-like strategy that improves optimization, allows our method to scale to large sample sizes. Our method also accounts for underestimation of posterior variance due to variational inference by providing an approximate posterior sampling approach based on Bayesian bootstrap ideas and spike-and-slab priors with random shrinkage targets. Besides accurate uncertainty quantification, this approach is capable of producing novel cluster size based imaging statistics, such as credible intervals of cluster size, and measures of reliability of cluster occurrence. Lastly, we validate our results via simulation studies and an application to the UK Biobank, a large-scale lesion mapping study with a sample size of 40,000 subjects. *(abstract via PubMed)*
SiM Diagnostics & Prediction High-Dimensional Machine Learning · 发表于 2024-01-08
Zoë S. Dunias, Ben Van Calster, Dirk Timmerman, Anne‐Laure Boulesteix, Maarten van Smeden
摘要
Tuning hyperparameters, such as the regularization parameter in Ridge or Lasso regression, is often aimed at improving the predictive performance of risk prediction models. In this study, various hyperparameter tuning procedures for clinical prediction models were systematically compared and evaluated in low‐dimensional data. The focus was on out‐of‐sample predictive performance (discrimination, calibration, and overall prediction error) of risk prediction models developed using Ridge, Lasso, Elastic Net, or Random Forest. The influence of sample size, number of predictors and events fraction on performance of the hyperparameter tuning procedures was studied using extensive simulations. The results indicate important differences between tuning procedures in calibration performance, while generally showing similar discriminative performance. The one‐standard‐error rule for tuning applied to cross‐validation (1SE CV) often resulted in severe miscalibration. Standard non‐repeated and repeated cross‐validation (both 5‐fold and 10‐fold) performed similarly well and outperformed the other tuning procedures. Bootstrap showed a slight tendency to more severe miscalibration than standard cross‐validation‐based tuning procedures. Differences between tuning procedures were larger for smaller sample sizes, lower events fractions and fewer predictors. These results imply that the choice of tuning procedure can have a profound influence on the predictive performance of prediction models. The results support the application of standard 5‐fold or 10‐fold cross‐validation that minimizes out‐of‐sample prediction error. Despite an increased computational burden, we found no clear benefit of repeated over non‐repeated cross‐validation for hyperparameter tuning. We warn against the potentially detrimental effects on model calibration of the popular 1SE CV rule for tuning prediction models in low‐dimensional settings.
SiM Testing & Uncertainty Nonparametric & Semiparametric · 发表于 2024-01-06
Fang Wan, Wei Liu, Frank Bretz
摘要
Regression modeling is the workhorse of statistics and there is a vast literature on estimation of the regression function. It has been realized in recent years that in regression analysis the ultimate aim may be the estimation of a level set of the regression function, ie, the set of covariate values for which the regression function exceeds a predefined level, instead of the estimation of the regression function itself. The published work on estimation of the level set has thus far focused mainly on nonparametric regression, especially on point estimation. In this article, the construction of confidence sets for the level set of linear regression is considered. In particular, level upper, lower and two‐sided confidence sets are constructed for the normal‐error linear regression. It is shown that these confidence sets can be easily constructed from the corresponding level simultaneous confidence bands. It is also pointed out that the construction method is readily applicable to other parametric regression models where the mean response depends on a linear predictor through a monotonic link function, which include generalized linear models, linear mixed models and generalized linear mixed models. Therefore, the method proposed in this article is widely applicable. Simulation studies with both linear and generalized linear models are conducted to assess the method and real examples are used to illustrate the method.
JASA Other · 发表于 2024-01-05
Giacomo Bormetti
摘要
Andreas Kyprianou and Juan Carlos Pardo offer an exceptional exploration of the captivating realms of stable Lévy processes. The book provides a comprehensive overview of stable processes and their... *(abstract via OpenAlex)*
JASA High-Dimensional · 发表于 2024-01-05
Wei Zhong, Zhuoxi Li, Wenwen Guo, Hengjian Cui
摘要
We propose a new measure of dependence between a categorical random variable and a random vector with potentially high dimensions, named semi-distance correlation. It is an interesting extension of distance correlation to accommodate the information of the categorical random variable. It equals zero if and only if the categorical random variable and the other random vector are independent. Two important applications of semi-distance correlation are considered. First, we develop a semi-distance independence test between a categorical random variable and a random vector and derive its asymptotic distributions. When the dimension of the random vector tends to infinity, we derive the explicit asymptotic normal distribution of the test statistic under the null hypothesis, which allows us to compute p-values in an efficient and fast way for high dimensional data. Second, we propose to use the semi-distance correlation as a marginal utility between the response and a group of covariates to do groupwise variable screening for ultrahigh dimensional classification problems. The sure screening property has also been established. Monte Carlo simulations and a real data application are presented to demonstrate the excellent finite sample property of the proposed procedures. A new R package semidist is also developed to implement the proposed methods. Supplementary materials for this article are available online. *(abstract via OpenAlex)*
JASA Causal Inference · 发表于 2024-01-05
Chan Park, Guanhua Chen, Menggang Yu, Hyunseung Kang
摘要
When developing policies for prevention of infectious diseases, policymakers often set specific, outcome-oriented targets to achieve. For example, when developing a vaccine allocation policy, policymakers may want to distribute them so that at least a certain fraction of individuals in a census block are disease-free and spillover effects due to interference within blocks are accounted for. The article proposes methods to estimate a block-level treatment policy that achieves a predefined, outcome-oriented target while accounting for spillover effects due to interference. Our policy, the minimum resource threshold policy (MRTP), suggests the minimum fraction of treated units required within a block to meet or exceed the target level of the outcome. We estimate the MRTP from empirical risk minimization using a novel, nonparametric, doubly robust loss function. We then characterize statistical properties of the estimated MRTP in terms of the excess risk bound. We apply our methodology to design a water, sanitation, and hygiene allocation policy for Senegal with the goal of increasing the proportion of households with no children experiencing diarrhea to a level exceeding a specified threshold. Our policy outperforms competing policies and offers new approaches to design allocation policies, especially in international development for communicable diseases. Supplementary materials for this article are available online. *(abstract via OpenAlex)*
JASA Causal Inference · 发表于 2024-01-05
Ting Ye
摘要
Fundamentals of Causal Inference: With R, by Babette Brumback provides an excellent introduction to causal inference methods and their implementations using R. The book is well-written, with many t... *(abstract via OpenAlex)*
JASA Optimization & Learning Theory Bayesian Methods · 发表于 2024-01-05
Niloy Biswas, Lester Mackey
摘要
Markov chain Monte Carlo (MCMC) provides asymptotically consistent estimates of intractable posterior expectations as the number of iterations tends to infinity. However, in large data applications, MCMC can be computationally expensive per iteration. This has catalyzed interest in approximating MCMC in a manner that improves computational speed per iteration but does not produce asymptotically consistent estimates. In this article, we propose estimators based on couplings of Markov chains to assess the quality of such asymptotically biased sampling methods. The estimators give empirical upper bounds of the Wasserstein distance between the limiting distribution of the asymptotically biased sampling method and the original target distribution of interest. We establish theoretical guarantees for our upper bounds and show that our estimators can remain effective in high dimensions. We apply our quality measures to stochastic gradient MCMC, variational Bayes, and Laplace approximations for tall data and to approximate MCMC for Bayesian logistic regression in 4500 dimensions and Bayesian linear regression in 50,000 dimensions. *(abstract via OpenAlex)*
JASA Other · 发表于 2024-01-04
Patrick M. LeBlanc, David Banks, Linhui Fu, Mingyan Li, Zhengyu Tang, Qiuyi Wu
摘要
Recommender systems are the engine of online advertising. Not only do they suggest movies, music, or romantic partners, but they also are used to select which advertisements to show to users. This paper reviews the basics of recommender system methodology and then looks at the emerging arena of active recommender systems. *(abstract via OpenAlex)*
JASA High-Dimensional · 发表于 2024-01-04
Siyi Deng, Yang Ning, Jiwei Zhao, Heping Zhang
摘要
We consider the estimation problem in high-dimensional semi-supervised learning. Our goal is to investigate when and how the unlabeled data can be exploited to improve the estimation of the regression parameters of linear model in light of the fact that such linear models may be misspecified in data analysis. We first establish the minimax lower bound for parameter estimation in the semi-supervised setting, and show that this lower bound cannot be achieved by supervised estimators using the labeled data only. We propose an optimal semi-supervised estimator that can attain this lower bound and therefore improves the supervised estimators, provided that the conditional mean function can be consistently estimated with a proper rate. We further propose a safe semi-supervised estimator. We view it safe, because this estimator is always at least as good as the supervised estimators. We also extend our idea to the aggregation of multiple semi-supervised estimators caused by different misspecifications of the conditional mean function. Extensive numerical simulations and a real data analysis are conducted to illustrate our theoretical results. *(abstract via PubMed)*
JASA Other · 发表于 2024-01-04
Xiaoyang Wu, Yuyang Huo, Haojie Ren, Changliang Zou
摘要
In the big data era, subsampling or sub-data selection techniques are often adopted to extract a fraction of informative individuals from the massive data. Existing subsampling algorithms focus mainly on obtaining a representative subset to achieve the best estimation accuracy under a given class of models. In this article, we consider a semi-supervised setting wherein a small or moderate sized “labeled” data is available in addition to a much larger sized “unlabeled” data. The goal is to sample from the unlabeled data with a given budget to obtain informative individuals that are characterized by their unobserved responses. We propose an optimal subsampling procedure that is able to maximize the diversity of the selected subsample and control the false selection rate (FSR) simultaneously, allowing us to explore reliable information as much as possible. The key ingredients of our method are the use of predictive inference for quantifying the uncertainty of response predictions and a reformulation of the objective into a constrained optimization problem. We show that the proposed method is asymptotically optimal in the sense that the diversity of the subsample converges to its oracle counterpart with FSR control. Numerical simulations and a real-data example validate the superior performance of the proposed strategy. Supplementary materials for this article are available online. *(abstract via OpenAlex)*
JASA Causal Inference Networks & Graphs · 发表于 2024-01-04
Tadao Hoshino, Takahide Yanagi
摘要
–We consider a causal inference model in which individuals interact in a social network and they may not comply with the assigned treatments. In particular, we suppose that the form of network interference is unknown to researchers. To estimate meaningful causal parameters in this situation, we introduce a new concept of exposure mapping, which summarizes potentially complicated spillover effects into a fixed dimensional statistic of instrumental variables. We investigate identification conditions for the intention-to-treat effects and the average treatment effects for compliers, while explicitly considering the possibility of misspecification of exposure mapping. Based on our identification results, we develop nonparametric estimation procedures via inverse probability weighting. Their asymptotic properties, including consistency and asymptotic normality, are investigated using an approximate neighborhood interference framework. For an empirical illustration, we apply our method to experimental data on the anti-conflict intervention school program. The proposed methods are readily available with the companion R package latenetwork. *(abstract via OpenAlex)*
JASA Causal Inference High-Dimensional Testing & Uncertainty · 发表于 2024-01-04
Yinrui Sun, Li Ma, Yin Xia
摘要
Motivated by the simultaneous association analysis with the presence of latent confounders, this article studies the large-scale hypothesis testing problem for the high-dimensional confounded linear models with both non-asymptotic and asymptotic false discovery control. Such model covers a wide range of practical settings where both the response and the predictors may be confounded. In the presence of the high-dimensional predictors and the unobservable confounders, the simultaneous inference with provable guarantees becomes highly challenging, and the unknown strong dependence among the confounded covariates makes the challenge even more pronounced. This article first introduces a decorrelating procedure that shrinks the confounding effect and weakens the correlations among the predictors, then performs debiasing under the decorrelated design based on some biased initial estimator. Following that, an asymptotic normality result for the debiased estimator is established and standardized test statistics are then constructed. Furthermore, a simultaneous inference procedure is proposed to identify significant associations, and both the finite-sample and asymptotic false discovery bounds are provided. The non-asymptotic result is general and model-free, and is of independent interest. We also prove that, under minimal signal strength condition, all associations can be successfully detected with probability tending to one. Simulation and real data studies are carried out to evaluate the performance of the proposed approach and compare it with other competing methods. Supplementary materials for this article are available online. *(abstract via OpenAlex)*
Econometrica Other · 发表于 2024
Yeon-Koo Che, Jinwoo Kim, Fuhito Kojima, Christopher Thomas Ryan
摘要
We give two characterizations of Pareto optimality via “near” weighted utilitarian welfare maximization. One characterization sequentially maximizes utilitarian welfare functions using a finite sequence of nonnegative and eventually positive welfare weights. The other maximizes a utilitarian welfare function with a certain class of positive hyperreal weights. The social welfare ordering represented by these “near” weighted utilitarian welfare criteria is characterized by the standard axioms for weighted utilitarianism under a suitable weakening of the continuity axiom.
JMLR High-Dimensional Optimization & Learning Theory · 发表于 2024
Zezhi Wang, Junxian Zhu, Xueqin Wang, Jin Zhu, Huiyang Pen, Peng Chen, Anran Wang, Xiaoke Zhang
摘要
Applying iterative solvers on sparsity-constrained optimization (SCO) requires tedious mathematical deduction and careful programming/debugging that hinders these solvers' broad impact. In the paper, the library skscope is introduced to overcome such an obstacle. With skscope, users can solve the SCO by just programming the objective function. The convenience of skscope is demonstrated through two examples in the paper, where sparse linear regression and trend filtering are addressed with just four lines of code. More importantly, skscope's efficient implementation allows state-of-the-art solvers to quickly attain the sparse solution regardless of the high dimensionality of parameter space. Numerical experiments reveal the available solvers in skscope can achieve up to 80x speedup on the competing relaxation solutions obtained via the benchmarked convex solver. skscope is published on the Python Package Index (PyPI) and Conda, and its source code is available at: https://github.com/abess-team/skscope.
JMLR Spatial & Time Series · 发表于 2024
Moritz Wolter, Felix Blanke, Jochen Garcke, Charles Tapley Hoyt
摘要
The fast wavelet transform is an essential workhorse in signal processing. Wavelets are local in the spatial- or temporal- and the frequency-domain. This property enables frequency domain analysis while preserving some spatiotemporal information. Until recently, wavelets rarely appeared in the machine learning literature. We provide the PyTorch Wavelet Toolbox to make wavelet methods more accessible to the deep learning community. Our PyTorch Wavelet Toolbox is well documented. A pip package is installable with `pip install ptwt`.
JMLR Bayesian Methods Networks & Graphs Causal Inference · 发表于 2024
Ankur Ankan, Johannes Textor
摘要
Bayesian Networks (BNs) are used in various fields for modeling, prediction, and decision making. pgmpy is a python package that provides a collection of algorithms and tools to work with BNs and related models. It implements algorithms for structure learning, parameter estimation, approximate and exact inference, causal inference, and simulations. These implementations focus on modularity and easy extensibility to allow users to quickly modify/add to existing algorithms, or to implement new algorithms for different use cases. pgmpy is released under the MIT License; the source code is available at: https://github.com/pgmpy/pgmpy, and the documentation at: https://pgmpy.org.
JMLR Spatial & Time Series Machine Learning · 发表于 2024
Matthew Middlehurst, Ali Ismail-Fawaz, Antoine Guillaume, Christopher Holder, David Guijo-Rubio, Guzal Bulatova, Leonidas Tsaprounis, Lukasz Mentel, Martin Walter, Patrick Schäfer, Anthony Bagnall
摘要
aeon is a unified Python 3 library for all machine learning tasks involving time series. The package contains modules for time series forecasting, classification, extrinsic regression and clustering, as well as a variety of utilities, transformations and distance measures designed for time series data. aeon also has a number of experimental modules for tasks such as anomaly detection, similarity search and segmentation. aeon follows the scikit-learn API as much as possible to help new users and enable easy integration of aeon estimators with useful tools such as model selection and pipelines. It provides a broad library of time series algorithms, including efficient implementations of the very latest advances in research. Using a system of optional dependencies, aeon integrates a wide variety of packages into a single interface while keeping the core framework with minimal dependencies. The package is distributed under the 3-Clause BSD license and is available at https://github.com/aeon-toolkit/aeon.
JMLR Optimization & Learning Theory · 发表于 2024
Yuefang Lian, Xiao Wang, Dachuan Xu, Zhongrui Zhao
摘要
In this paper, we study approximation algorithms for several classes of DR-submodular optimization problems, where DR is short for diminishing return. Following a newly introduced algorithm framework for zeroth-order stochastic approximation methods, we first propose algorithms {\bf CG-ZOSA} and {\bf RG-ZOSA} for smooth DR-submodular optimization based on the coordinate-wise gradient estimator and the randomized gradient estimator, respectively. Our theoretical analysis proves that \rm{\bf{CG-ZOSA}} can reach a solution whose expected objective value exceeds $(1-e^{-1}-\epsilon^{2})$OPT$-\epsilon$ after $\mathcal{O}(\epsilon^{-2})$ iterations and $\mathcal{O}(N^{2/3}d\epsilon^{-2})$ oracle calls, where $d$ represents the problem dimension. On the other hand, \rm{\bf{RG-ZOSA}} improves the approximation ratio to $(1-e^{-1}-\epsilon^{2}/d)$ while maintaining the same overall oracle complexity. For non-smooth up-concave maximization problems, we propose a novel auxiliary function based on a smoothed objective function and introduce the \rm{\bf{NZOSA}} algorithm. This algorithm achieves an approximation ratio of $(1-e^{-1}-\epsilon \ln \epsilon^{-1}- \epsilon^{2}\ln \epsilon^{-1})$ with $\mathcal{O}(d\epsilon^{-2})$ iterations and $\mathcal{O}(N^{2/3}d^{3/2} \epsilon^{-3})$ oracle calls. We also extend \rm{\bf{NZOSA}} to handle a class of robust DR-submodular maximization problems. To validate the effectiveness of our proposed algorithms, we conduct experiments on both synthetic and real-world problems. The results demonstrate the superior performance and efficiency of our methods in solving DR-submodular optimization problems.
JMLR Networks & Graphs · 发表于 2024
Pan Zhou, Xingyu Xie, Zhouchen Lin, Kim-Chuan Toh, Shuicheng Yan
摘要
Training deep networks on large-scale datasets is computationally challenging. This work explores the problem of “how to accelerate adaptive gradient algorithms in a general manner", and proposes an effective Weight-decay-Integrated Nesterov acceleration (Win) to accelerate adaptive algorithms. Taking AdamW and Adam as examples, per iteration, we construct a dynamical loss that combines the vanilla training loss and a dynamic regularizer inspired by proximal point method, and respectively minimize the first- and second-order Taylor approximations of dynamical loss to update variable. This yields our Win acceleration that uses a conservative step and an aggressive step to update, and linearly combines these two updates for acceleration. Next, we extend Win into Win2 which uses multiple aggressive update steps for faster convergence. Then we apply Win and Win2 to the popular LAMB and SGD optimizers. Our transparent derivation could provide insights for other accelerated methods and their integration into adaptive algorithms. Besides, we theoretically justify the faster convergence of Win- and Win2-accelerated AdamW, Adam and LAMB to their non-accelerated counterparts. Experimental results demonstrates the faster convergence speed and superior performance of our Win- and Win2-accelerated AdamW, Adam, LAMB and SGD over their vanilla counterparts on vision classification and language modeling tasks.
JMLR Machine Learning High-Dimensional Networks & Graphs · 发表于 2024
Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Hao Bai, Yuexiang Zhai, Benjamin D. Haeffele, Yi Ma
摘要
In this paper, we contend that a natural objective of representation learning is to compress and transform the distribution of the data, say sets of tokens, towards a low-dimensional Gaussian mixture supported on incoherent subspaces. The goodness of such a representation can be evaluated by a principled measure, called sparse rate reduction, that simultaneously maximizes the intrinsic information gain and extrinsic sparsity of the learned representation. From this perspective, popular deep network architectures, including transformers, can be viewed as realizing iterative schemes to optimize this measure. Particularly, we derive a transformer block from alternating optimization on parts of this objective: the multi-head self-attention operator compresses the representation by implementing an approximate gradient descent step on the coding rate of the features, and the subsequent multi-layer perceptron sparsifies the features. This leads to a family of white-box transformer-like deep network architectures, named CRATE, which are mathematically fully interpretable. We show, by way of a novel connection between denoising and compression, that the inverse to the aforementioned compressive encoding can be realized by the same class of CRATE architectures. Thus, the so-derived white-box architectures are universal to both encoders and decoders. Experiments show that these networks, despite their simplicity, indeed learn to compress and sparsify representations of large-scale real-world image and text datasets, and achieve strong performance across different settings: ViT, MAE, DINO, BERT, and GPT2. We believe the proposed computational framework demonstrates great potential in bridging the gap between theory and practice of deep learning, from a unified perspective of data compression.
Econometrica Other · 发表于 2024
Matthieu Gomez, Émilien Gouin-Bonenfant
摘要
We study the effect of interest rates on wealth inequality. While lower rates decrease the growth rate of rentiers, they also increase the growth rate of entrepreneurs by making it cheaper to raise capital. To understand which effect dominates, we derive a sufficient statistic for the effect of interest rates on the Pareto exponent of the wealth distribution: it depends on the lifetime equity and debt issuance rate of individuals in the right tail of the wealth distribution. We estimate this sufficient statistic using new data on the trajectory of top fortunes in the U.S. Overall, we find that the secular decline in interest rates (or more generally of required rates of returns) can account for about 40% of the rise in Pareto inequality; that is, the degree to which the super rich pulled ahead relative to the rich.
JMLR Optimization & Learning Theory · 发表于 2024
Rentian Yao, Xiaohui Chen, Yun Yang
摘要
Motivated by approximation Bayesian computation using mean-field variational approximation and the computation of equilibrium in multi-species systems with cross-interaction, this paper investigates the composite geodesically convex optimization problem over multiple distributions. The objective functional under consideration is composed of a convex potential energy on a product of Wasserstein spaces and a sum of convex self-interaction and internal energies associated with each distribution. To efficiently solve this problem, we introduce the Wasserstein Proximal Coordinate Gradient (WPCG) algorithms with parallel, sequential, and random update schemes. Under a quadratic growth (QG) condition that is weaker than the usual strong convexity requirement on the objective functional, we show that WPCG converges exponentially fast to the unique global optimum. In the absence of the QG condition, WPCG is still demonstrated to converge to the global optimal solution, albeit at a slower polynomial rate. Numerical results for both motivating examples are consistent with our theoretical findings.
Econometrica Econometrics & Economics · 发表于 2024
Shubhdeep Deb, Jan Eeckhout, Aseem Patel, Lawrence Warren
摘要
We propose a theory of how market power affects wage inequality. We ask how goods and labor market power jointly determine the level of wages, the skill premium, and wage inequality. We then use detailed microdata from the U.S. Census Bureau between 1997 and 2016 to estimate the parameters of labor supply, technology, and the market structure. We find that a less competitive market structure lowers the average wage of high‐skilled workers by 11.3%, and of low‐skilled workers by 12.2%, contributes 8.1% to the rise in the skill premium, and accounts for 54.8% of the increase in between‐establishment wage variance.
JMLR Machine Learning Networks & Graphs · 发表于 2024
Siddharth Roheda, Hamid Krim, Bo Jiang
摘要
The importance of inference in Machine Learning (ML) has led to an explosive number of different proposals, particularly in Deep Learning. In an attempt to reduce the complexity of Convolutional Neural Networks, we propose a Volterra filter-inspired Network architecture. This architecture introduces controlled non-linearities in the form of interactions between the delayed input samples of data. We propose a cascaded implementation of Volterra Filtering so as to significantly reduce the number of parameters required to carry out the same classification task as that of a conventional Neural Network. We demonstrate an efficient parallel implementation of this Volterra Neural Network (VNN), along with its remarkable performance while retaining a relatively simpler and potentially more tractable structure. Furthermore, we show a rather sophisticated adaptation of this network to nonlinearly fuse the RGB (spatial) information and the Optical Flow (temporal) information of a video sequence for action recognition. The proposed approach is evaluated on UCF-101 and HMDB-51 datasets for action recognition, and is shown to outperform state of the art CNN approaches.