2024 年的期刊文章 · 第 26 页

共 1515 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2024-01-11 到 2024。

按期刊、按主题看 →

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 页

JASA Testing & Uncertainty · 发表于 2024-01-11

Split Knockoffs for Multiple Comparisons: Controlling the Directional False Discovery Rate

Yang Cao, Xinwei Sun, Yuan Yao

摘要

Multiple comparisons in hypothesis testing often encounter structural constraints in various applications. For instance, in structural Magnetic Resonance Imaging for Alzheimer’s Disease, the focus extends beyond examining atrophic brain regions to include comparisons of anatomically adjacent regions. These constraints can be modeled as linear transformations of parameters, where the sign patterns play a crucial role in estimating directional effects. This class of problems, encompassing total variations, wavelet transforms, fused LASSO, trend filtering, and more, presents an open challenge in effectively controlling the directional false discovery rate. In this article, we propose an extended Split Knockoff method specifically designed to address the control of directional false discovery rate under linear transformations. Our proposed approach relaxes the stringent linear manifold constraint to its neighborhood, employing a variable splitting technique commonly used in optimization. This methodology yields an orthogonal design that benefits both power and directional false discovery rate control. By incorporating a sample splitting scheme, we achieve effective control of the directional false discovery rate, with a notable reduction to zero as the relaxed neighborhood expands. To demonstrate the efficacy of our method, we conduct simulation experiments and apply it to two real-world scenarios: Alzheimer’s Disease analysis and human age comparisons. Supplementary materials for this article are available online. *(abstract via OpenAlex)*

SiM Causal Inference Clinical Trials Bayesian Methods · 发表于 2024-01-11

Confidence distributions for treatment effects in clinical trials: Posteriors without priors

Ian C. Marschner

摘要

An attractive feature of using a Bayesian analysis for a clinical trial is that knowledge and uncertainty about the treatment effect is summarized in a posterior probability distribution. Researchers often find probability statements about treatment effects highly intuitive and the fact that this is not accommodated in frequentist inference is a disadvantage. At the same time, the requirement to specify a prior distribution in order to obtain a posterior distribution is sometimes an artificial process that may introduce subjectivity or complexity into the analysis. This paper considers a compromise involving confidence distributions, which are probability distributions that summarize uncertainty about the treatment effect without the need for a prior distribution and in a way that is fully compatible with frequentist inference. The concept of a confidence distribution provides a posterior–like probability distribution that is distinct from, but exists in tandem with, the relative frequency interpretation of probability used in frequentist inference. Although they have been discussed for decades, confidence distributions are not well known among clinical trial statisticians and the goal of this paper is to discuss their use in analyzing treatment effects from randomized trials. As well as providing an introduction to confidence distributions, some illustrative examples relevant to clinical trials are presented, along with various case studies based on real clinical trials. It is recommended that trial statisticians consider presenting confidence distributions for treatment effects when reporting analyses of clinical trials.

JRSSB Spatial & Time Series Machine Learning Networks & Graphs · 发表于 2024-01-11

Automatic change-point detection in time series via deep learning

Jie Li, Paul Fearnhead, Piotr Fryzlewicz, Tengyao Wang

摘要

Detecting change points in data is challenging because of the range of possible types of change and types of behaviour of data when there is no change. Statistically efficient methods for detecting a change will depend on both of these features, and it can be difficult for a practitioner to develop an appropriate detection method for their application of interest. We show how to automatically generate new offline detection methods based on training a neural network. Our approach is motivated by many existing tests for the presence of a change point being representable by a simple neural network, and thus a neural network trained with sufficient data should have performance at least as good as these methods. We present theory that quantifies the error rate for such an approach, and how it depends on the amount of training data. Empirical results show that, even with limited training data, its performance is competitive with the standard cumulative sum (CUSUM) based classifier for detecting a change in mean when the noise is independent and Gaussian, and can substantially outperform it in the presence of auto-correlated or heavy-tailed noise. Our method also shows strong results in detecting and localizing changes in activity based on accelerometer data.

JRSSB Causal Inference · 发表于 2024-01-10

Optimal individualized treatment rule for combination treatments under budget constraints

Qi Xu, Haoda Fu, Annie Qu

摘要

The individualized treatment rule (ITR), which recommends an optimal treatment based on individual characteristics, has drawn considerable interest from many areas such as precision medicine, personalized education, and personalized marketing. Existing ITR estimation methods mainly adopt 1 of 2 or more treatments. However, a combination of multiple treatments could be more powerful in various areas. In this paper, we propose a novel double encoder model (DEM) to estimate the ITR for combination treatments. The proposed double encoder model is a nonparametric model which not only flexibly incorporates complex treatment effects and interaction effects among treatments but also improves estimation efficiency via the parameter-sharing feature. In addition, we tailor the estimated ITR to budget constraints through a multi-choice knapsack formulation, which enhances our proposed method under restricted-resource scenarios. In theory, we provide the value reduction bound with or without budget constraints, and an improved convergence rate with respect to the number of treatments under the DEM. Our simulation studies show that the proposed method outperforms the existing ITR estimation in various settings. We also demonstrate the superior performance of the proposed method in patient-derived xenograft data that recommends optimal combination treatments to shrink the tumour size of the colorectal cancer.

SiM Longitudinal Data Networks & Graphs Latent Variables & Mixtures · 发表于 2024-01-10

Joint modeling of association networks and longitudinal biomarkers: An application to childhood obesity

Andrea Cremaschi, Maria De Iorio, Narasimhan Kothandaraman, Fabian Yap, Mya Thway Tint, Johan Eriksson

摘要

The prevalence of chronic non‐communicable diseases such as obesity has noticeably increased in the last decade. The study of these diseases in early life is of paramount importance in determining their course in adult life and in supporting clinical interventions. Recently, attention has been drawn to approaches that study the alteration of metabolic pathways in obese children. In this work, we propose a novel joint modeling approach for the analysis of growth biomarkers and metabolite associations, to unveil metabolic pathways related to childhood obesity. Within a Bayesian framework, we flexibly model the temporal evolution of growth trajectories and metabolic associations through the specification of a joint nonparametric random effect distribution, with the main goal of clustering subjects, thus identifying risk sub‐groups. Growth profiles as well as patterns of metabolic associations determine the clustering structure. Inclusion of risk factors is straightforward through the specification of a regression term. We demonstrate the proposed approach on data from the Growing Up in Singapore Towards healthy Outcomes cohort study, based in Singapore. Posterior inference is obtained via a tailored MCMC algorithm, involving a nonparametric prior with mixed support. Our analysis has identified potential key pathways in obese children that allow for the exploration of possible molecular mechanisms associated with childhood obesity.

JRSSB Testing & Uncertainty · 发表于 2024-01-10

Integrative conformal p-values for out-of-distribution testing with labelled outliers

Ziyi Liang, Matteo Sesia, Wenguang Sun

摘要

This paper presents a conformal inference method for out-of-distribution testing that leverages side information from labelled outliers, which are commonly underutilized or even discarded by conventional conformal p-values. This solution is practical and blends inductive and transductive inference strategies to adaptively weight conformal p-values, while also automatically leveraging the most powerful model from a collection of one-class and binary classifiers. Further, this approach leads to rigorous false discovery rate control in multiple testing when combined with a conditional calibration strategy. Extensive numerical simulations show that the proposed method outperforms existing approaches.

JASA Clinical Trials Causal Inference Machine Learning · 发表于 2024-01-09

Model-Robust and Efficient Covariate Adjustment for Cluster-Randomized Experiments

Bingkai Wang, Chan Park, Dylan S. Small, Fan Li

摘要

Cluster-randomized experiments are increasingly used to evaluate interventions in routine practice conditions, and researchers often adopt model-based methods with covariate adjustment in the statistical analyses. However, the validity of model-based covariate adjustment remains unclear when the working models are misspecified, leading to ambiguity of estimands and risk of bias. In this article, we first adapt two model-based methods-generalized estimating equations and linear mixed models-with weighted g-computation to achieve robust inference for cluster-average and individual-average treatment effects. To further overcome the limitations of model-based covariate adjustment methods, we propose efficient estimators for each estimand that allow for flexible covariate adjustment and additionally address cluster size variation dependent on treatment assignment and other cluster characteristics. Such cluster size variations often occur post-randomization and, if ignored, can lead to bias of model-based estimators. For our proposed covariate-adjusted estimators, we prove that when the nuisance functions are consistently estimated by machine learning algorithms, the estimators are consistent, asymptotically normal, and efficient. When the nuisance functions are estimated via parametric working models, the estimators are triply-robust. Simulation studies and analyses of three real-world cluster-randomized experiments demonstrate that the proposed methods are superior to existing alternatives. *(abstract via PubMed)*

SMMR Survival Analysis Functional Data Diagnostics & Prediction · 发表于 2024-01-09

Dynamic prediction of survival using multivariate functional principal component analysis: A strict landmarking approach

Daniel Gomon, Hein Putter, Marta Fiocco, Mirko Signorelli

摘要

Dynamically predicting patient survival probabilities using longitudinal measurements has become of great importance with routine data collection becoming more common. Many existing models utilize a multi-step landmarking approach for this problem, mostly due to its ease of use and versatility but unfortunately most fail to do so appropriately. In this article we make use of multivariate functional principal component analysis to summarize the available longitudinal information, and employ a Cox proportional hazards model for prediction. Additionally, we consider a centred functional principal component analysis procedure in an attempt to remove the natural variation incurred by the difference in age of the considered subjects. We formalize the difference between a ‘relaxed’ landmarking approach where only validation data is landmarked and a ‘strict’ landmarking approach where both the training and validation data are landmarked. We show that a relaxed landmarking approach fails to effectively use the information contained in the longitudinal outcomes, thereby producing substantially worse prediction accuracy than a strict landmarking approach.

JASA Testing & Uncertainty · 发表于 2024-01-09

Bootstrap Inference in the Presence of Bias

Giuseppe Cavaliere, Sílvia Gonçalves, Morten Ørregaard Nielsen, Edoardo Zanelli

摘要

We consider bootstrap inference for estimators which are (asymptotically) biased. We show that, even when the bias term cannot be consistently estimated, valid inference can be obtained by proper implementations of the bootstrap. Specifically, we show that the prepivoting approach of Beran (1987, 1988), originally proposed to deliver higher-order refinements, restores bootstrap validity by transforming the original bootstrap p-value into an asymptotically uniform random variable. We propose two different implementations of prepivoting (plug-in and double bootstrap), and provide general high-level conditions that imply validity of bootstrap inference. To illustrate the practical relevance and implementation of our results, we discuss five examples: (i) inference on a target parameter based on model averaging; (ii) ridge-type regularized estimators; (iii) nonparametric regression; (iv) a location model for infinite variance data; and (v) dynamic panel data models. *(abstract via OpenAlex)*

JASA Other · 发表于 2024-01-09

Balancing Covariates in Randomized Experiments with the Gram–Schmidt Walk Design

Christopher Harshaw, Fredrik Sävje, Daniel A. Spielman, Peng Zhang

摘要

The design of experiments involves a compromise between covariate balance and robustness.This article provides a formalization of this tradeoff and describes an experimental design that allows experimenters to navigate it.The design is specified by a robustness parameter that bounds the worst-case mean squared error of an estimator of the average treatment effect.Subject to the experimenter's desired level of robustness, the design aims to simultaneously balance all linear functions of potentially many covariates.Less robustness allows for more balance.We show that the mean squared error of the estimator is bounded in finite samples by the minimum of the loss function of an implicit ridge regression of the potential outcomes on the covariates.Asymptotically, the design perfectly balances all linear functions of a growing number of covariates with a diminishing reduction in robustness, effectively allowing experimenters to escape the compromise between balance and robustness in large samples.Finally, we describe conditions that ensure asymptotic normality and provide a conservative variance estimator, which facilitate the construction of asymptotically valid confidence intervals.Supplementary materials for this article are available online. *(abstract via OpenAlex)*

JASA Bayesian Methods Spatial & Time Series High-Dimensional · 发表于 2024-01-08

Bayesian Lesion Estimation with a Structured Spike-and-Slab Prior

Anna Menacher, Thomas E. Nichols, Chris Holmes, Habib Ganjgahi

摘要

Neural demyelination and brain damage accumulated in white matter appear as hyperintense areas on T2-weighted MRI scans in the form of lesions. Modeling binary images at the population level, where each voxel represents the existence of a lesion, plays an important role in understanding aging and inflammatory diseases. We propose a scalable hierarchical Bayesian spatial model, called BLESS, capable of handling binary responses by placing continuous spike-and-slab mixture priors on spatially-varying parameters and enforcing spatial dependency on the parameter dictating the amount of sparsity within the probability of inclusion. The use of mean-field variational inference with dynamic posterior exploration, which is an annealing-like strategy that improves optimization, allows our method to scale to large sample sizes. Our method also accounts for underestimation of posterior variance due to variational inference by providing an approximate posterior sampling approach based on Bayesian bootstrap ideas and spike-and-slab priors with random shrinkage targets. Besides accurate uncertainty quantification, this approach is capable of producing novel cluster size based imaging statistics, such as credible intervals of cluster size, and measures of reliability of cluster occurrence. Lastly, we validate our results via simulation studies and an application to the UK Biobank, a large-scale lesion mapping study with a sample size of 40,000 subjects. *(abstract via PubMed)*

SiM Diagnostics & Prediction High-Dimensional Machine Learning · 发表于 2024-01-08

A comparison of hyperparameter tuning procedures for clinical prediction models: A simulation study

Zoë S. Dunias, Ben Van Calster, Dirk Timmerman, Anne‐Laure Boulesteix, Maarten van Smeden

摘要

Tuning hyperparameters, such as the regularization parameter in Ridge or Lasso regression, is often aimed at improving the predictive performance of risk prediction models. In this study, various hyperparameter tuning procedures for clinical prediction models were systematically compared and evaluated in low‐dimensional data. The focus was on out‐of‐sample predictive performance (discrimination, calibration, and overall prediction error) of risk prediction models developed using Ridge, Lasso, Elastic Net, or Random Forest. The influence of sample size, number of predictors and events fraction on performance of the hyperparameter tuning procedures was studied using extensive simulations. The results indicate important differences between tuning procedures in calibration performance, while generally showing similar discriminative performance. The one‐standard‐error rule for tuning applied to cross‐validation (1SE CV) often resulted in severe miscalibration. Standard non‐repeated and repeated cross‐validation (both 5‐fold and 10‐fold) performed similarly well and outperformed the other tuning procedures. Bootstrap showed a slight tendency to more severe miscalibration than standard cross‐validation‐based tuning procedures. Differences between tuning procedures were larger for smaller sample sizes, lower events fractions and fewer predictors. These results imply that the choice of tuning procedure can have a profound influence on the predictive performance of prediction models. The results support the application of standard 5‐fold or 10‐fold cross‐validation that minimizes out‐of‐sample prediction error. Despite an increased computational burden, we found no clear benefit of repeated over non‐repeated cross‐validation for hyperparameter tuning. We warn against the potentially detrimental effects on model calibration of the popular 1SE CV rule for tuning prediction models in low‐dimensional settings.

JRSSB Machine Learning Spatial & Time Series Networks & Graphs 讨论 / 评论 · 发表于 2024-01-07

M.N.M. van Lieshout and C. Lu’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Marie-Colette van Lieshout, Changqing Lu

摘要

We congratulate Professors Li, Fearnhead, Fryzlewics, and Wang on their fine work that represents classic test statistics for change-point detection as a neural network-based classifier and develops improved offline detection algorithms for historical, labelled data. The paper focuses on real-valued observation vectors in a temporal regression model with training data being either labelled historical data or obtained by simulation from a model. In many applications, though, the data consist of both spatial and temporal components. For instance, the observations may take the form of a series of point patterns (e.g. mapped tree locations at different census times) or a single observation from a spatio-temporal point pattern (e.g. occurrence locations and times of fire incidents). For the latter, detection of changes in intensity in a model-based Bayesian test setting was investigated by Altieri et al. (2015). Do the authors believe that an adapted neural network approach could be competitive in this context? A complication would be that the change-point is due to complex changes in inter-point interaction for which neither a known model nor labelled historical data is available. Do the authors see a way forward here? Machine learning ideas could benefit spatio-temporal statistical practice more widely. Specifically for point pattern analysis, Lu, Van Lieshout et al. (2023) employed random forest importance scores for variable selection, whilst Jalilian and Mateu (2023) trained neural networks on simulated data to distinguish spatial structural differences. A similar motivation as that of Li et al. is seen in point process intensity estimation. Usually, the intensity function is assumed to be log-linear in spatial and temporal covariates. Lu, Guan, et al. (2023) proposed a tree-based model, XGBoostPP, which forms the intensity function based on a covariate vector z(s) as Here, fk{z(s)} are tree predictors that output the response on the leaf where a covariate value z(s) lies in. For model fitting, we customized a penalized weighted Poisson log-likelihood loss function where X denotes the point process on S and Ω(fk) is proportional to the L1-norm of leaf responses. The tree structures and corresponding leaf responses are optimized iteratively; the weights w are calculated based on the estimated inhomogeneous K-function. The classic log-linear intensity function can be represented as a reparameterized XGBoostPP; neural networks may offer alternatives. In the reverse direction, the stable with respect to iterations of tessellations tessellation (Nagel & Weiss, 2005) from spatial statistics can be used for partitioning responses (cf. Ge et al., 2019). This research was funded by the Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO) for the project ‘Data driven risk management for fire services’ (18004). *(abstract via OpenAlex)*

JRSSB Machine Learning Spatial & Time Series 讨论 / 评论 · 发表于 2024-01-07

Ayla Jungbluth and Johannes Lederer’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Ayla Jungbluth, Johannes Lederer

摘要

We have read the paper ‘Automatic Change-Point Detection in Time Series Via Deep Learning’ by Jie Li, Paul Fearnhead, Piotr Fryzlewicz, and Tengyao Wang with great interest. We congratulate the authors for an important and forward-looking contribution. We agree that time-series data often have change points that can degrade the prediction quality considerably. The paper’s use of deep learning to detect change points is, therefore, definitely interesting. Besides, our own work shows that change-point detection is also vital for time-series pipelines that are based on deep learning themselves (Jungbluth & Lederer, 2023). We have also shown that change points can be accounted in deep learning in a surprisingly simple and effective way. Indeed, most deep-learning-based forecasting methods learn by taking batches from the time-series data. In essence, our BatchCP method selects batches from the time series such that change points are avoided while we are still able to use the entire time series as data. For this, BatchCP needs to know where the change points are before the training begins. Based on this information, the batches are then deemed suitable or not as illustrated in Figure 1. Visualization of the change-point method BatchCP, taken from the mentioned reference. It shows an example of a batch selection around a change-point. The points represent time-series data. The first area shows a batch created by choosing t = 1 as the start index, and with a selected batch size of s = 6, the batch contains the points t = 1 to t = 6. The change-point is located at t = 7 and is therefore outside the detected batch. The batch is therefore permissible. The area in the middle shows a prohibited batch. The start index of this batch is t = 4 and the end index at t = 9, so the change-point is located inside the batch and is not allowed in training. A new batch must be found. The area to the right indicates another permissible batch, since the change-point lies outside of it. BatchCP is agnostic to the way change points are detected and, therefore, should work very well with the method proposed in the discussed paper. Indeed, the combination of BatchCP and the proposed method in their paper would equip modern architectures such as DeepAR and transformers with a quite automated approach to deal with change points. Our numerical studies still need to be completed, but the initial results look promising already. *(abstract via OpenAlex)*

SiM Testing & Uncertainty Nonparametric & Semiparametric · 发表于 2024-01-06

Confidence sets for a level set in linear regression

Fang Wan, Wei Liu, Frank Bretz

摘要

Regression modeling is the workhorse of statistics and there is a vast literature on estimation of the regression function. It has been realized in recent years that in regression analysis the ultimate aim may be the estimation of a level set of the regression function, ie, the set of covariate values for which the regression function exceeds a predefined level, instead of the estimation of the regression function itself. The published work on estimation of the level set has thus far focused mainly on nonparametric regression, especially on point estimation. In this article, the construction of confidence sets for the level set of linear regression is considered. In particular, level upper, lower and two‐sided confidence sets are constructed for the normal‐error linear regression. It is shown that these confidence sets can be easily constructed from the corresponding level simultaneous confidence bands. It is also pointed out that the construction method is readily applicable to other parametric regression models where the mean response depends on a linear predictor through a monotonic link function, which include generalized linear models, linear mixed models and generalized linear mixed models. Therefore, the method proposed in this article is widely applicable. Simulation studies with both linear and generalized linear models are conducted to assess the method and real examples are used to illustrate the method.

JASA Other · 发表于 2024-01-05

Stable Lévy Processes via Lamperti-Type Representations

Giacomo Bormetti

摘要

Andreas Kyprianou and Juan Carlos Pardo offer an exceptional exploration of the captivating realms of stable Lévy processes. The book provides a comprehensive overview of stable processes and their... *(abstract via OpenAlex)*

JASA High-Dimensional · 发表于 2024-01-05

Semi-Distance Correlation and Its Applications

Wei Zhong, Zhuoxi Li, Wenwen Guo, Hengjian Cui

摘要

We propose a new measure of dependence between a categorical random variable and a random vector with potentially high dimensions, named semi-distance correlation. It is an interesting extension of distance correlation to accommodate the information of the categorical random variable. It equals zero if and only if the categorical random variable and the other random vector are independent. Two important applications of semi-distance correlation are considered. First, we develop a semi-distance independence test between a categorical random variable and a random vector and derive its asymptotic distributions. When the dimension of the random vector tends to infinity, we derive the explicit asymptotic normal distribution of the test statistic under the null hypothesis, which allows us to compute p-values in an efficient and fast way for high dimensional data. Second, we propose to use the semi-distance correlation as a marginal utility between the response and a group of covariates to do groupwise variable screening for ultrahigh dimensional classification problems. The sure screening property has also been established. Monte Carlo simulations and a real data application are presented to demonstrate the excellent finite sample property of the proposed procedures. A new R package semidist is also developed to implement the proposed methods. Supplementary materials for this article are available online. *(abstract via OpenAlex)*

JRSSB Spatial & Time Series Causal Inference · 发表于 2024-01-05

Policy evaluation for temporal and/or spatial dependent experiments

Shikai Luo, Ying Yang, Chengchun Shi, Fang Yao, Jieping Ye, Hongtu Zhu

摘要

The aim of this article is to establish a causal link between the policies implemented by technology companies and the outcomes they yield within intricate temporal and/or spatial dependent experiments. We propose a novel temporal/spatio-temporal Varying Coefficient Decision Process model, capable of effectively capturing the evolving treatment effects in situations characterized by temporal and/or spatial dependence. Our methodology encompasses the decomposition of the average treatment effect into the direct effect (DE) and the indirect effect (IE). We subsequently devise comprehensive procedures for estimating and making inferences about both DE and IE. Additionally, we provide a rigorous analysis of the statistical properties of these procedures, such as asymptotic power. To substantiate the effectiveness of our approach, we carry out extensive simulations and real data analyses.

JASA Causal Inference · 发表于 2024-01-05

Minimum Resource Threshold Policy Under Partial Interference

Chan Park, Guanhua Chen, Menggang Yu, Hyunseung Kang

摘要

When developing policies for prevention of infectious diseases, policymakers often set specific, outcome-oriented targets to achieve. For example, when developing a vaccine allocation policy, policymakers may want to distribute them so that at least a certain fraction of individuals in a census block are disease-free and spillover effects due to interference within blocks are accounted for. The article proposes methods to estimate a block-level treatment policy that achieves a predefined, outcome-oriented target while accounting for spillover effects due to interference. Our policy, the minimum resource threshold policy (MRTP), suggests the minimum fraction of treated units required within a block to meet or exceed the target level of the outcome. We estimate the MRTP from empirical risk minimization using a novel, nonparametric, doubly robust loss function. We then characterize statistical properties of the estimated MRTP in terms of the excess risk bound. We apply our methodology to design a water, sanitation, and hygiene allocation policy for Senegal with the goal of increasing the proportion of households with no children experiencing diarrhea to a level exceeding a specified threshold. Our policy outperforms competing policies and offers new approaches to design allocation policies, especially in international development for communicable diseases. Supplementary materials for this article are available online. *(abstract via OpenAlex)*

JRSSB Machine Learning 讨论 / 评论 · 发表于 2024-01-05

James Jackson’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

James Jackson

摘要

James Jackson; James Jackson’s Contribution to the Discussion of ‘The Discussion Meeting on Probabilistic and Statistical Aspects of Machine Learning’, Journal *(abstract via OpenAlex)*

JASA Causal Inference · 发表于 2024-01-05

Fundamentals of Causal Inference: With R

Ting Ye

摘要

Fundamentals of Causal Inference: With R, by Babette Brumback provides an excellent introduction to causal inference methods and their implementations using R. The book is well-written, with many t... *(abstract via OpenAlex)*

JASA Optimization & Learning Theory Bayesian Methods · 发表于 2024-01-05

Bounding Wasserstein Distance with Couplings

Niloy Biswas, Lester Mackey

摘要

Markov chain Monte Carlo (MCMC) provides asymptotically consistent estimates of intractable posterior expectations as the number of iterations tends to infinity. However, in large data applications, MCMC can be computationally expensive per iteration. This has catalyzed interest in approximating MCMC in a manner that improves computational speed per iteration but does not produce asymptotically consistent estimates. In this article, we propose estimators based on couplings of Markov chains to assess the quality of such asymptotically biased sampling methods. The estimators give empirical upper bounds of the Wasserstein distance between the limiting distribution of the asymptotically biased sampling method and the original target distribution of interest. We establish theoretical guarantees for our upper bounds and show that our estimators can remain effective in high dimensions. We apply our quality measures to stochastic gradient MCMC, variational Bayes, and Laplace approximations for tall data and to approximate MCMC for Bayesian logistic regression in 4500 dimensions and Bayesian linear regression in 50,000 dimensions. *(abstract via OpenAlex)*

JASA Other · 发表于 2024-01-04

Recommender Systems: A Review

Patrick M. LeBlanc, David Banks, Linhui Fu, Mingyan Li, Zhengyu Tang, Qiuyi Wu

摘要

Recommender systems are the engine of online advertising. Not only do they suggest movies, music, or romantic partners, but they also are used to select which advertisements to show to users. This paper reviews the basics of recommender system methodology and then looks at the emerging arena of active recommender systems. *(abstract via OpenAlex)*

JASA High-Dimensional · 发表于 2024-01-04

Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning

Siyi Deng, Yang Ning, Jiwei Zhao, Heping Zhang

摘要

We consider the estimation problem in high-dimensional semi-supervised learning. Our goal is to investigate when and how the unlabeled data can be exploited to improve the estimation of the regression parameters of linear model in light of the fact that such linear models may be misspecified in data analysis. We first establish the minimax lower bound for parameter estimation in the semi-supervised setting, and show that this lower bound cannot be achieved by supervised estimators using the labeled data only. We propose an optimal semi-supervised estimator that can attain this lower bound and therefore improves the supervised estimators, provided that the conditional mean function can be consistently estimated with a proper rate. We further propose a safe semi-supervised estimator. We view it safe, because this estimator is always at least as good as the supervised estimators. We also extend our idea to the aggregation of multiple semi-supervised estimators caused by different misspecifications of the conditional mean function. Extensive numerical simulations and a real data analysis are conducted to illustrate our theoretical results. *(abstract via PubMed)*

JASA Other · 发表于 2024-01-04

Optimal Subsampling via Predictive Inference

Xiaoyang Wu, Yuyang Huo, Haojie Ren, Changliang Zou

摘要

In the big data era, subsampling or sub-data selection techniques are often adopted to extract a fraction of informative individuals from the massive data. Existing subsampling algorithms focus mainly on obtaining a representative subset to achieve the best estimation accuracy under a given class of models. In this article, we consider a semi-supervised setting wherein a small or moderate sized “labeled” data is available in addition to a much larger sized “unlabeled” data. The goal is to sample from the unlabeled data with a given budget to obtain informative individuals that are characterized by their unobserved responses. We propose an optimal subsampling procedure that is able to maximize the diversity of the selected subsample and control the false selection rate (FSR) simultaneously, allowing us to explore reliable information as much as possible. The key ingredients of our method are the use of predictive inference for quantifying the uncertainty of response predictions and a reformulation of the objective into a constrained optimization problem. We show that the proposed method is asymptotically optimal in the sense that the diversity of the subsample converges to its oracle counterpart with FSR control. Numerical simulations and a real-data example validate the superior performance of the proposed strategy. Supplementary materials for this article are available online. *(abstract via OpenAlex)*

JASA Causal Inference Networks & Graphs · 发表于 2024-01-04

Causal Inference with Noncompliance and Unknown Interference

Tadao Hoshino, Takahide Yanagi

摘要

–We consider a causal inference model in which individuals interact in a social network and they may not comply with the assigned treatments. In particular, we suppose that the form of network interference is unknown to researchers. To estimate meaningful causal parameters in this situation, we introduce a new concept of exposure mapping, which summarizes potentially complicated spillover effects into a fixed dimensional statistic of instrumental variables. We investigate identification conditions for the intention-to-treat effects and the average treatment effects for compliers, while explicitly considering the possibility of misspecification of exposure mapping. Based on our identification results, we develop nonparametric estimation procedures via inverse probability weighting. Their asymptotic properties, including consistency and asymptotic normality, are investigated using an approximate neighborhood interference framework. For an empirical illustration, we apply our method to experimental data on the anti-conflict intervention school program. The proposed methods are readily available with the companion R package latenetwork. *(abstract via OpenAlex)*

JASA Causal Inference High-Dimensional Testing & Uncertainty · 发表于 2024-01-04

A Decorrelating and Debiasing Approach to Simultaneous Inference for High-Dimensional Confounded Models

Yinrui Sun, Li Ma, Yin Xia

摘要

Motivated by the simultaneous association analysis with the presence of latent confounders, this article studies the large-scale hypothesis testing problem for the high-dimensional confounded linear models with both non-asymptotic and asymptotic false discovery control. Such model covers a wide range of practical settings where both the response and the predictors may be confounded. In the presence of the high-dimensional predictors and the unobservable confounders, the simultaneous inference with provable guarantees becomes highly challenging, and the unknown strong dependence among the confounded covariates makes the challenge even more pronounced. This article first introduces a decorrelating procedure that shrinks the confounding effect and weakens the correlations among the predictors, then performs debiasing under the decorrelated design based on some biased initial estimator. Following that, an asymptotic normality result for the debiased estimator is established and standardized test statistics are then constructed. Furthermore, a simultaneous inference procedure is proposed to identify significant associations, and both the finite-sample and asymptotic false discovery bounds are provided. The non-asymptotic result is general and model-free, and is of independent interest. We also prove that, under minimal signal strength condition, all associations can be successfully detected with probability tending to one. Simulation and real data studies are carried out to evaluate the performance of the proposed approach and compare it with other competing methods. Supplementary materials for this article are available online. *(abstract via OpenAlex)*

JRSSB Machine Learning Networks & Graphs 讨论 / 评论 · 发表于 2024-01-02

Seconder of the vote of thanks and contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Christopher Nemeth

摘要

I congratulate the authors of these two papers for their insightful and significant contributions to addressing the statistical aspects of machine learning. In this contribution to these discussion papers, I will make a short comment which covers both papers and then provide some separate thoughts on each paper. These two papers are quite different in their focus; however, a common thread between them is the use of neural networks, and in particular deep neural networks, to augment part of the modelling process. In the case of Li et al. (2022), neural networks are used to convert the changepoint problem into a supervised learning problem, and in the case of Benton et al. (2022), neural networks are used to approximate the intractable score function. A better understanding of the statistical properties of neural networks is an ongoing area of research (Anthony et al., 1999; Bartlett et al., 2019), but their application has become widespread within the artificial intelligence and machine learning communities. Within the statistics community, we can look back 30 years to the discussion paper of Ripley (1994) to see how neural networks can be used to solve classification problems. Interestingly, although the Ripley (1994) and Li et al. (2022) papers are very different in their focus, they both utilize neural networks to solve a classification problem and derive similar theoretical results regarding neural network complexity in terms of Vapnik–Chervonenkis (VC)-dimension bounds. The authors comment in the conclusion to their paper that for applied statisticians trying to model a changepoint problem: ‘…they need to understand what type of change is sought, be able to characterize it mathematically, find a satisfactory stochastic model for the data, formulate the appropriate statistic, and fine-tune its parameters’. However, if instead of trying to model the data-generating process, a neural network is used, then does this not lead to the same challenges? What type of neural network should be used? How deep should the network be? As an illustration of this point, let us consider the example from Section 5 of the paper, with ρ=0 under Scenario 1. Figure 1 presented here is similar to Figure 2 in Li et al. (2022) where I have changed the activation function from ReLU (left) to SeLU (middle) and sigmoid (right). Under these different activation functions, we can see that for sufficiently large N, the neural network-based approaches are superior to the CUSUM statistics, but for small N the average misclassification error rate can be quite different depending on the choice of the activation function. What guidelines are available for practitioners to ensure that they use the best neural network architecture? Is it easier to choose an appropriate neural network than it is to choose a statistical model that directly models the time series data? Scenario 1: n=100,N∈{100,200,…,700},ρ=0⁠. ReLU (left), SeLU (middle), and Sigmoid (right) activation functions. The authors focus their empirical presentation on the statistical improvements of their neural network-based approach when compared against the CUSUM statistic. However, there is no presentation of the difference in computational cost between the CUSUM and neural network approaches. Would it be more reasonable to report the misclassification error rate scaled by computational time? This would be interesting to consider because if it takes twice as long, or perhaps longer, to fit the neural network compared to the CUSUM test, then what percentage of improvement should we expect to see from the neural network as a result of the increased computational complexity? The general denoising framework proposed in this paper is an important contribution that allows the class of diffusion generative models, which are rapidly growing in popularity, to be applied to non-Euclidean spaces. The examples in the paper are directed towards sampling from non-standard spaces; however, the first example, which considers an approximate Bayesian inference model, is on Rd and provides a nice illustration of how these diffusion models are applied on simpler problems. What is of particular interest in Section 6.1 is that the results presented are not significantly better than many existing approximate Bayesian computation algorithms. By some standards, this is quite a simple example as there are only four parameters to be learnt, and yet the neural network (a multilayer perceptron) used by the authors to approximate the score function has 1.9 million parameters (see Appendix J.1). It does seem somewhat paradoxical that in order to approximate a four-dimensional distribution it is necessary to estimate 1.9 million parameters in a neural network. Furthermore, the observed dataset is of size 250, which leads to interesting questions around how feasible it is to learn a large number of parameters from a neural network with small datasets. Is it possible to know what type of neural network should be used for a particular generative problem, e.g. images, text, etc.? Or how large the network needs to be in order to achieve high levels of statistical accuracy? In conclusion, these two papers provide stimulating contributions to the field of statistical machine learning and open up many interesting avenues of future research in these areas, it is a pleasure to second the vote of thanks. The vote of thanks was passed by acclamation. *(abstract via OpenAlex)*

Econometrica Other · 发表于 2024

“Near” Weighted Utilitarian Characterizations of Pareto Optima

Yeon-Koo Che, Jinwoo Kim, Fuhito Kojima, Christopher Thomas Ryan

摘要

We give two characterizations of Pareto optimality via “near” weighted utilitarian welfare maximization. One characterization sequentially maximizes utilitarian welfare functions using a finite sequence of nonnegative and eventually positive welfare weights. The other maximizes a utilitarian welfare function with a certain class of positive hyperreal weights. The social welfare ordering represented by these “near” weighted utilitarian welfare criteria is characterized by the standard axioms for weighted utilitarianism under a suitable weakening of the continuity axiom.

JMLR High-Dimensional Optimization & Learning Theory · 发表于 2024

skscope: Fast Sparsity-Constrained Optimization in Python

Zezhi Wang, Junxian Zhu, Xueqin Wang, Jin Zhu, Huiyang Pen, Peng Chen, Anran Wang, Xiaoke Zhang

摘要

Applying iterative solvers on sparsity-constrained optimization (SCO) requires tedious mathematical deduction and careful programming/debugging that hinders these solvers' broad impact. In the paper, the library skscope is introduced to overcome such an obstacle. With skscope, users can solve the SCO by just programming the objective function. The convenience of skscope is demonstrated through two examples in the paper, where sparse linear regression and trend filtering are addressed with just four lines of code. More importantly, skscope's efficient implementation allows state-of-the-art solvers to quickly attain the sparse solution regardless of the high dimensionality of parameter space. Numerical experiments reveal the available solvers in skscope can achieve up to 80x speedup on the competing relaxation solutions obtained via the benchmarked convex solver. skscope is published on the Python Package Index (PyPI) and Conda, and its source code is available at: https://github.com/abess-team/skscope.

JMLR Spatial & Time Series · 发表于 2024

ptwt - The PyTorch Wavelet Toolbox

Moritz Wolter, Felix Blanke, Jochen Garcke, Charles Tapley Hoyt

摘要

The fast wavelet transform is an essential workhorse in signal processing. Wavelets are local in the spatial- or temporal- and the frequency-domain. This property enables frequency domain analysis while preserving some spatiotemporal information. Until recently, wavelets rarely appeared in the machine learning literature. We provide the PyTorch Wavelet Toolbox to make wavelet methods more accessible to the deep learning community. Our PyTorch Wavelet Toolbox is well documented. A pip package is installable with `pip install ptwt`.

JMLR Bayesian Methods Networks & Graphs Causal Inference · 发表于 2024

pgmpy: A Python Toolkit for Bayesian Networks

Ankur Ankan, Johannes Textor

摘要

Bayesian Networks (BNs) are used in various fields for modeling, prediction, and decision making. pgmpy is a python package that provides a collection of algorithms and tools to work with BNs and related models. It implements algorithms for structure learning, parameter estimation, approximate and exact inference, causal inference, and simulations. These implementations focus on modularity and easy extensibility to allow users to quickly modify/add to existing algorithms, or to implement new algorithms for different use cases. pgmpy is released under the MIT License; the source code is available at: https://github.com/pgmpy/pgmpy, and the documentation at: https://pgmpy.org.

JMLR Spatial & Time Series Machine Learning · 发表于 2024

aeon: a Python Toolkit for Learning from Time Series

Matthew Middlehurst, Ali Ismail-Fawaz, Antoine Guillaume, Christopher Holder, David Guijo-Rubio, Guzal Bulatova, Leonidas Tsaprounis, Lukasz Mentel, Martin Walter, Patrick Schäfer, Anthony Bagnall

摘要

aeon is a unified Python 3 library for all machine learning tasks involving time series. The package contains modules for time series forecasting, classification, extrinsic regression and clustering, as well as a variety of utilities, transformations and distance measures designed for time series data. aeon also has a number of experimental modules for tasks such as anomaly detection, similarity search and segmentation. aeon follows the scikit-learn API as much as possible to help new users and enable easy integration of aeon estimators with useful tools such as model selection and pipelines. It provides a broad library of time series algorithms, including efficient implementations of the very latest advances in research. Using a system of optional dependencies, aeon integrates a wide variety of packages into a single interface while keeping the core framework with minimal dependencies. The package is distributed under the 3-Clause BSD license and is available at https://github.com/aeon-toolkit/aeon.

JMLR Optimization & Learning Theory · 发表于 2024

Zeroth-order Stochastic Approximation Algorithms for DR-submodular Optimization

Yuefang Lian, Xiao Wang, Dachuan Xu, Zhongrui Zhao

摘要

In this paper, we study approximation algorithms for several classes of DR-submodular optimization problems, where DR is short for diminishing return. Following a newly introduced algorithm framework for zeroth-order stochastic approximation methods, we first propose algorithms {\bf CG-ZOSA} and {\bf RG-ZOSA} for smooth DR-submodular optimization based on the coordinate-wise gradient estimator and the randomized gradient estimator, respectively. Our theoretical analysis proves that \rm{\bf{CG-ZOSA}} can reach a solution whose expected objective value exceeds $(1-e^{-1}-\epsilon^{2})$OPT$-\epsilon$ after $\mathcal{O}(\epsilon^{-2})$ iterations and $\mathcal{O}(N^{2/3}d\epsilon^{-2})$ oracle calls, where $d$ represents the problem dimension. On the other hand, \rm{\bf{RG-ZOSA}} improves the approximation ratio to $(1-e^{-1}-\epsilon^{2}/d)$ while maintaining the same overall oracle complexity. For non-smooth up-concave maximization problems, we propose a novel auxiliary function based on a smoothed objective function and introduce the \rm{\bf{NZOSA}} algorithm. This algorithm achieves an approximation ratio of $(1-e^{-1}-\epsilon \ln \epsilon^{-1}- \epsilon^{2}\ln \epsilon^{-1})$ with $\mathcal{O}(d\epsilon^{-2})$ iterations and $\mathcal{O}(N^{2/3}d^{3/2} \epsilon^{-3})$ oracle calls. We also extend \rm{\bf{NZOSA}} to handle a class of robust DR-submodular maximization problems. To validate the effectiveness of our proposed algorithms, we conduct experiments on both synthetic and real-world problems. The results demonstrate the superior performance and efficiency of our methods in solving DR-submodular optimization problems.

JMLR Networks & Graphs · 发表于 2024

Win: Weight-Decay-Integrated Nesterov Acceleration for Faster Network Training

Pan Zhou, Xingyu Xie, Zhouchen Lin, Kim-Chuan Toh, Shuicheng Yan

摘要

Training deep networks on large-scale datasets is computationally challenging. This work explores the problem of “how to accelerate adaptive gradient algorithms in a general manner", and proposes an effective Weight-decay-Integrated Nesterov acceleration (Win) to accelerate adaptive algorithms. Taking AdamW and Adam as examples, per iteration, we construct a dynamical loss that combines the vanilla training loss and a dynamic regularizer inspired by proximal point method, and respectively minimize the first- and second-order Taylor approximations of dynamical loss to update variable. This yields our Win acceleration that uses a conservative step and an aggressive step to update, and linearly combines these two updates for acceleration. Next, we extend Win into Win2 which uses multiple aggressive update steps for faster convergence. Then we apply Win and Win2 to the popular LAMB and SGD optimizers. Our transparent derivation could provide insights for other accelerated methods and their integration into adaptive algorithms. Besides, we theoretically justify the faster convergence of Win- and Win2-accelerated AdamW, Adam and LAMB to their non-accelerated counterparts. Experimental results demonstrates the faster convergence speed and superior performance of our Win- and Win2-accelerated AdamW, Adam, LAMB and SGD over their vanilla counterparts on vision classification and language modeling tasks.

JMLR Machine Learning High-Dimensional Networks & Graphs · 发表于 2024

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?

Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Hao Bai, Yuexiang Zhai, Benjamin D. Haeffele, Yi Ma

摘要

In this paper, we contend that a natural objective of representation learning is to compress and transform the distribution of the data, say sets of tokens, towards a low-dimensional Gaussian mixture supported on incoherent subspaces. The goodness of such a representation can be evaluated by a principled measure, called sparse rate reduction, that simultaneously maximizes the intrinsic information gain and extrinsic sparsity of the learned representation. From this perspective, popular deep network architectures, including transformers, can be viewed as realizing iterative schemes to optimize this measure. Particularly, we derive a transformer block from alternating optimization on parts of this objective: the multi-head self-attention operator compresses the representation by implementing an approximate gradient descent step on the coding rate of the features, and the subsequent multi-layer perceptron sparsifies the features. This leads to a family of white-box transformer-like deep network architectures, named CRATE, which are mathematically fully interpretable. We show, by way of a novel connection between denoising and compression, that the inverse to the aforementioned compressive encoding can be realized by the same class of CRATE architectures. Thus, the so-derived white-box architectures are universal to both encoders and decoders. Experiments show that these networks, despite their simplicity, indeed learn to compress and sparsify representations of large-scale real-world image and text datasets, and achieve strong performance across different settings: ViT, MAE, DINO, BERT, and GPT2. We believe the proposed computational framework demonstrates great potential in bridging the gap between theory and practice of deep learning, from a unified perspective of data compression.

Econometrica Other · 发表于 2024

Wealth Inequality in a Low Rate Environment

Matthieu Gomez, Émilien Gouin-Bonenfant

摘要

We study the effect of interest rates on wealth inequality. While lower rates decrease the growth rate of rentiers, they also increase the growth rate of entrepreneurs by making it cheaper to raise capital. To understand which effect dominates, we derive a sufficient statistic for the effect of interest rates on the Pareto exponent of the wealth distribution: it depends on the lifetime equity and debt issuance rate of individuals in the right tail of the wealth distribution. We estimate this sufficient statistic using new data on the trajectory of top fortunes in the U.S. Overall, we find that the secular decline in interest rates (or more generally of required rates of returns) can account for about 40% of the rise in Pareto inequality; that is, the degree to which the super rich pulled ahead relative to the rich.

JMLR Optimization & Learning Theory · 发表于 2024

Wasserstein Proximal Coordinate Gradient Algorithms

Rentian Yao, Xiaohui Chen, Yun Yang

摘要

Motivated by approximation Bayesian computation using mean-field variational approximation and the computation of equilibrium in multi-species systems with cross-interaction, this paper investigates the composite geodesically convex optimization problem over multiple distributions. The objective functional under consideration is composed of a convex potential energy on a product of Wasserstein spaces and a sum of convex self-interaction and internal energies associated with each distribution. To efficiently solve this problem, we introduce the Wasserstein Proximal Coordinate Gradient (WPCG) algorithms with parallel, sequential, and random update schemes. Under a quadratic growth (QG) condition that is weaker than the usual strong convexity requirement on the objective functional, we show that WPCG converges exponentially fast to the unique global optimum. In the absence of the QG condition, WPCG is still demonstrated to converge to the global optimal solution, albeit at a slower polynomial rate. Numerical results for both motivating examples are consistent with our theoretical findings.

Econometrica Econometrics & Economics · 发表于 2024

Walras–Bowley Lecture: Market Power and Wage Inequality

Shubhdeep Deb, Jan Eeckhout, Aseem Patel, Lawrence Warren

摘要

We propose a theory of how market power affects wage inequality. We ask how goods and labor market power jointly determine the level of wages, the skill premium, and wage inequality. We then use detailed microdata from the U.S. Census Bureau between 1997 and 2016 to estimate the parameters of labor supply, technology, and the market structure. We find that a less competitive market structure lowers the average wage of high‐skilled workers by 11.3%, and of low‐skilled workers by 12.2%, contributes 8.1% to the rise in the skill premium, and accounts for 54.8% of the increase in between‐establishment wage variance.

JMLR Machine Learning Networks & Graphs · 发表于 2024

Volterra Neural Networks (VNNs)

Siddharth Roheda, Hamid Krim, Bo Jiang

摘要

The importance of inference in Machine Learning (ML) has led to an explosive number of different proposals, particularly in Deep Learning. In an attempt to reduce the complexity of Convolutional Neural Networks, we propose a Volterra filter-inspired Network architecture. This architecture introduces controlled non-linearities in the form of interactions between the delayed input samples of data. We propose a cascaded implementation of Volterra Filtering so as to significantly reduce the number of parameters required to carry out the same classification task as that of a conventional Neural Network. We demonstrate an efficient parallel implementation of this Volterra Neural Network (VNN), along with its remarkable performance while retaining a relatively simpler and potentially more tractable structure. Furthermore, we show a rather sophisticated adaptation of this network to nonlinearly fuse the RGB (spatial) information and the Optical Flow (temporal) information of a video sequence for action recognition. The proposed approach is evaluated on UCF-101 and HMDB-51 datasets for action recognition, and is shown to outperform state of the art CNN approaches.

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 页