2024 年的期刊文章 · 第 24 页

共 1515 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2024-02-10 到 2024-01-31。

按期刊、按主题看 →

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 页

SiM Clinical Trials Bayesian Methods · 发表于 2024-02-10

A flexible multi‐metric Bayesian framework for decision‐making in Phase II multi‐arm multi‐stage studies

Suzanne M. Dufault, Angela M. Crook, Katie Rolfe, Patrick P. J. Phillips

摘要

We propose a multi‐metric flexible Bayesian framework to support efficient interim decision‐making in multi‐arm multi‐stage phase II clinical trials. Multi‐arm multi‐stage phase II studies increase the efficiency of drug development, but early decisions regarding the futility or desirability of a given arm carry considerable risk since sample sizes are often low and follow‐up periods may be short. Further, since intermediate outcomes based on biomarkers of treatment response are rarely perfect surrogates for the primary outcome and different trial stakeholders may have different levels of risk tolerance, a single hypothesis test is insufficient for comprehensively summarizing the state of the collected evidence. We present a Bayesian framework comprised of multiple metrics based on point estimates, uncertainty, and evidence towards desired thresholds (a Target Product Profile) for (1) ranking of arms and (2) comparison of each arm against an internal control. Using a large public‐private partnership targeting novel TB arms as a motivating example, we find via simulation study that our multi‐metric framework provides sufficient confidence for decision‐making with sample sizes as low as 30 patients per arm, even when intermediate outcomes have only moderate correlation with the primary outcome. Our reframing of trial design and the decision‐making procedure has been well‐received by research partners and is a practical approach to more efficient assessment of novel therapeutics.

SMMR Causal Inference Survival Analysis · 发表于 2024-02-08

Heterogeneous treatment effect estimation for observational data using model-based forests

Susanne Dandl, Andreas Bender, Torsten Hothorn

摘要

The estimation of heterogeneous treatment effects has attracted considerable interest in many disciplines, most prominently in medicine and economics. Contemporary research has so far primarily focused on continuous and binary responses where heterogeneous treatment effects are traditionally estimated by a linear model, which allows the estimation of constant or heterogeneous effects even under certain model misspecifications. More complex models for survival, count, or ordinal outcomes require stricter assumptions to reliably estimate the treatment effect. Most importantly, the noncollapsibility issue necessitates the joint estimation of treatment and prognostic effects. Model-based forests allow simultaneous estimation of covariate-dependent treatment and prognostic effects, but only for randomized trials. In this paper, we propose modifications to model-based forests to address the confounding issue in observational data. In particular, we evaluate an orthogonalization strategy originally proposed by Robinson (1988, Econometrica) in the context of model-based forests targeting heterogeneous treatment effect estimation in generalized linear models and transformation models. We found that this strategy reduces confounding effects in a simulated study with various outcome distributions. We demonstrate the practical aspects of heterogeneous treatment effect estimation for survival and ordinal outcomes by an assessment of the potentially heterogeneous effect of Riluzole on the progress of Amyotrophic Lateral Sclerosis.

SiM Spatial & Time Series Longitudinal Data · 发表于 2024-02-08

Generalized functional linear model with a point process predictor

Jiehuan Sun, Kuang‐Yao Lee

摘要

Point process data have become increasingly popular these days. For example, many of the data captured in electronic health records (EHR) are in the format of point process data. It is of great interest to study the association between a point process predictor and a scalar response using generalized functional linear regression models. Various generalized functional linear regression models have been developed under different settings in the past decades. However, existing methods can only deal with functional or longitudinal predictors, not point process predictors. In this article, we propose a novel generalized functional linear regression model for a point process predictor. Our proposed model is based on the joint modeling framework, where we adopt a log‐Gaussian Cox process model for the point process predictor and a generalized linear regression model for the outcome. We also develop a new algorithm for fast model estimation based on the Gaussian variational approximation method. We conduct extensive simulation studies to evaluate the performance of our proposed method and compare it to competing methods. The performance of our proposed method is further demonstrated on an EHR dataset of patients admitted into the intensive care units of the Beth Israel Deaconess Medical Center between 2001 and 2008.

SMMR Clinical Trials Bayesian Methods · 发表于 2024-02-07

Covariate adjustment in Bayesian adaptive randomized controlled trials

James Willard, Shirin Golchi, Erica EM Moodie

摘要

In conventional randomized controlled trials, adjustment for baseline values of covariates known to be at least moderately associated with the outcome increases the power of the trial. Recent work has shown a particular benefit for more flexible frequentist designs, such as information adaptive and adaptive multi-arm designs. However, covariate adjustment has not been characterized within the more flexible Bayesian adaptive designs, despite their growing popularity. We focus on a subclass of these which allow for early stopping at an interim analysis given evidence of treatment superiority. We consider both collapsible and non-collapsible estimands and show how to obtain posterior samples of marginal estimands from adjusted analyses. We describe several estimands for three common outcome types. We perform a simulation study to assess the impact of covariate adjustment using a variety of adjustment models in several different scenarios. This is followed by a real-world application of the compared approaches to a COVID-19 trial with a binary endpoint. For all scenarios, it is shown that covariate adjustment increases power and the probability of stopping the trials early, and decreases the expected sample sizes as compared to unadjusted analyses.

SMMR Clinical Trials Diagnostics & Prediction · 发表于 2024-02-07

A diagnostic phase III/IV seamless design to investigate the diagnostic accuracy and clinical effectiveness using the example of HEDOS and HEDOS II

Amra Pepić, Maria Stark, Tim Friede, Annette Kopp-Schneider, Silvia Calderazzo, Maria Reichert, Michael Wolf, Ulrich Wirth, Stefan Schopf, Antonia Zapf

摘要

The development process of medical devices can be streamlined by combining different study phases. Here, for a diagnostic medical device, we present the combination of confirmation of diagnostic accuracy (phase III) and evaluation of clinical effectiveness regarding patient-relevant endpoints (phase IV) using a seamless design. This approach is used in the Thyroid HEmorrhage DetectOr Study (HEDOS & HEDOS II) investigating a post-operative hemorrhage detector named ISAR-M THYRO® in patients after thyroid surgery. Data from the phase III trial are reused as external controls in the control group of the phase IV trial. An unblinded interim analysis is planned between the two study stages which includes a recalculation of the sample size for the phase IV part after completion of the first stage of the seamless design. The study concept presented here is the first seamless design proposed in the field of diagnostic studies. Hence, the aim of this work is to emphasize the statistical methodology as well as feasibility of the proposed design in relation to the planning and implementation of the seamless design. Seamless designs can accelerate the overall trial duration and increase its efficiency in terms of sample size and recruitment. However, careful planning addressing numerous methodological and procedural challenges is necessary for successful implementation as well as agreement with regulatory bodies.

SMMR Diagnostics & Prediction · 发表于 2024-02-06

Using a centered general linear model for detection of interactions among biomarkers

Tao Wang, Chien-Wei Lin

摘要

The dummy variable based general linear model (gLM) is commonly used to model categorical factors and their interactions. However, the main factors and their interactions in a general linear model are often correlated even when the factors are independently distributed. Alternatively, the classical two-way factorial analysis of variance (ANOVA) model can avoid the correlation between the main factors and their interactions when the main factors are independent. But the ANOVA model is hardly applicable to a regular linear regression model especially in the presence of other covariates due to constraints on its model parameters. In this study, a centered general linear model (cgLM) is proposed for modeling interactions between categorical factors based on their centered dummy variables. We show that the cgLM can avoid the correlation between the main factors and their interactions as the ANOVA model when the main factors are independent. Meanwhile, similar to gLM, it can be used in regular regression and fitted conveniently using the standard least square approach by choosing appropriate baselines to avoid constraints on its model parameters. The potential advantage of cgLM over gLM for detection of interactions in model building procedures is also illustrated and compared via a simulation study. Finally, the cgLM is applied to a postmortem brain gene expression data set.

SMMR Causal Inference · 发表于 2024-02-06

The augmented synthetic control method in public health and biomedical research

Taylor Krajewski, Michael Hudgens

摘要

Estimating treatment (or policy or intervention) effects on a single individual or unit has become increasingly important in health and biomedical sciences. One method to estimate these effects is the synthetic control method, which constructs a synthetic control, a weighted average of control units that best matches the treated unit’s pre-treatment outcomes and other relevant covariates. The intervention’s impact is then estimated by comparing the post-intervention outcomes of the treated unit and its synthetic control, which serves as a proxy for the counterfactual outcome had the treated unit not experienced the intervention. The augmented synthetic control method, a recent adaptation of the synthetic control method, relaxes some of the synthetic control method’s assumptions for broader applicability. While synthetic controls have been used in a variety of fields, their use in public health and biomedical research is more recent, and newer methods such as the augmented synthetic control method are underutilized. This paper briefly describes the synthetic control method and its application, explains the augmented synthetic control method and its differences from the synthetic control method, and estimates the effects of an antimalarial initiative in Mozambique using both the synthetic control method and the augmented synthetic control method to highlight the advantages of using the augmented synthetic control method to analyze the impact of interventions implemented in a single region.

SiM Survival Analysis High-Dimensional · 发表于 2024-02-06

Simultaneous variable selection and estimation for survival data via the Gaussian seamless‐L0$$ {L}_0 $$ penalty

Zili Liu, Hong Wang

摘要

We propose a new simultaneous variable selection and estimation procedure with the Gaussian seamless‐ (GSELO) penalty for Cox proportional hazard model and additive hazards model. The GSELO procedure shows good potential to improve the existing variable selection methods by taking strength from both best subset selection (BSS) and regularization. In addition, we develop an iterative algorithm to implement the proposed procedure in a computationally efficient way. Theoretically, we establish the convergence properties of the algorithm and asymptotic theoretical properties of the proposed procedure. Since parameter tuning is crucial to the performance of the GSELO procedure, we also propose an extended Bayesian information criteria (EBIC) parameter selector for the GSELO procedure. Simulated and real data studies have demonstrated the prediction performance and effectiveness of the proposed method over several state‐of‐the‐art methods.

SiM Other · 发表于 2024-02-06

Sensitivity analysis with iterative outlier detection for systematic reviews and meta‐analyses

Zhuo Meng, Jingshen Wang, Lifeng Lin, Chong Wu

摘要

Meta‐analysis is a widely used tool for synthesizing results from multiple studies. The collected studies are deemed heterogeneous when they do not share a common underlying effect size; thus, the factors attributable to the heterogeneity need to be carefully considered. A critical problem in meta‐analyses and systematic reviews is that outlying studies are frequently included, which can lead to invalid conclusions and affect the robustness of decision‐making. Outliers may be caused by several factors such as study selection criteria, low study quality, small‐study effects, and so on. Although outlier detection is well‐studied in the statistical community, limited attention has been paid to meta‐analysis. The conventional outlier detection method in meta‐analysis is based on a leave‐one‐study‐out procedure. However, when calculating a potentially outlying study's deviation, other outliers could substantially impact its result. This article proposes an iterative method to detect potential outliers, which reduces such an impact that could confound the detection. Furthermore, we adopt bagging to provide valid inference for sensitivity analyses of excluding outliers. Based on simulation studies, the proposed iterative method yields smaller bias and heterogeneity after performing a sensitivity analysis to remove the identified outliers. It also provides higher accuracy on outlier detection. Two case studies are used to illustrate the proposed method's real‐world performance.

SiM Longitudinal Data · 发表于 2024-02-06

Multivariate probit linear mixed models for multivariate longitudinal binary data

Kuo‐Jung Lee, Chanmin Kim, Jae Keun Yoo, Keunbaik Lee

摘要

When analyzing multivariate longitudinal binary data, we estimate the effects on the responses of the covariates while accounting for three types of complex correlations present in the data. These include the correlations within separate responses over time, cross‐correlations between different responses at different times, and correlations between different responses at each time point. The number of parameters thus increases quadratically with the dimension of the correlation matrix, making parameter estimation difficult; the estimated correlation matrix must also meet the positive definiteness constraint. The correlation matrix may additionally be heteroscedastic; however, the matrix structure is commonly considered to be homoscedastic and constrained, such as exchangeable or autoregressive with order one. These assumptions are overly strong, resulting in skewed estimates of the covariate effects on the responses. Hence, we propose probit linear mixed models for multivariate longitudinal binary data, where the correlation matrix is estimated using hypersphere decomposition instead of the strong assumptions noted above. Simulations and real examples are used to demonstrate the proposed methods. An open source R package, BayesMGLM, is made available on GitHub at https://github.com/kuojunglee/BayesMGLM/ with full documentation to produce the results.

SiM Causal Inference · 发表于 2024-02-05

Targeted learning in observational studies with multi‐valued treatments: An evaluation of antipsychotic drug treatment safety

Jason Poulos, Marcela Horvitz‐Lennon, Katya Zelevinsky, Tudor Cristea‐Platon, Thomas Huijskens, Pooja Tyagi, Jiaju Yan, Jordi Diaz, Sharon‐Lise Normand

摘要

We investigate estimation of causal effects of multiple competing (multi‐valued) treatments in the absence of randomization. Our work is motivated by an intention‐to‐treat study of the relative cardiometabolic risk of assignment to one of six commonly prescribed antipsychotic drugs in a cohort of nearly 39 000 adults with serious mental illnesses. Doubly‐robust estimators, such as targeted minimum loss‐based estimation (TMLE), require correct specification of either the treatment model or outcome model to ensure consistent estimation; however, common TMLE implementations estimate treatment probabilities using multiple binomial regressions rather than multinomial regression. We implement a TMLE estimator that uses multinomial treatment assignment and ensemble machine learning to estimate average treatment effects. Our multinomial implementation improves coverage, but does not necessarily reduce bias, relative to the binomial implementation in simulation experiments with varying treatment propensity overlap and event rates. Evaluating the causal effects of the antipsychotics on 3‐year diabetes risk or death, we find a safety benefit of moving from a second‐generation drug considered among the safest of the second‐generation drugs to an infrequently prescribed first‐generation drug known for having low cardiometabolic risk.

SiM Econometrics & Economics · 发表于 2024-02-05

Scaled average bioequivalence methods for highly variable drugs: Leveling‐off soft limits and the EMA's 2010 guideline (some ways to improve its type I error control)

Joel Muñoz, Jordi Ocaña, Rolando Suárez, Carolina Millapán

摘要

The regulatory EMA's reference scaled average bioequivalence (RSABE) approach for highly variable drugs suffers from some type I error control problems at the neighborhood of the 30% coefficient of variation (CV), where the bioequivalence (BE) limits change from constant to linearly scaled. This paper analyses BE inference methods based on the “Leveling‐off” (LO) soft sigmoid expanding BE limits that were proposed as an appealing surrogate for the EMA's limits and compares both approaches, on the replicated and partially replicated crossover designs. The initially proposed version of the LO method also has type I error inflation problems, albeit attenuated. But given its more mathematically regular character, it is more suitable for analytical corrections. Here we introduce two improvements over LO, one based on the application of Howe's method and the other based on correcting the estimation error. They further reduce the type I error inflation, although it does not disappear completely. Finally, the effect of heteroscedasticity on the above results is studied. It leads to inflation or deflation of the type I error, depending on the design and the type of heteroscedasticity (variability of the test product greater than that of the reference product or the opposite). The replicated design is much more stable against these effects than the partially replicated design and maintains these improvements much better.

SiM Clinical Trials · 发表于 2024-02-05

Multiply robust generalized estimating equations for cluster randomized trials with missing outcomes

Dustin J. Rabideau, Fan Li, Rui Wang

摘要

Generalized estimating equations (GEEs) provide a useful framework for estimating marginal regression parameters based on data from cluster randomized trials (CRTs), but they can result in inaccurate parameter estimates when some outcomes are informatively missing. Existing techniques to handle missing outcomes in CRTs rely on correct specification of a propensity score model, a covariate‐conditional mean outcome model, or require at least one of these two models to be correct, which can be challenging in practice. In this article, we develop new weighted GEEs to simultaneously estimate the marginal mean, scale, and correlation parameters in CRTs with missing outcomes, allowing for multiple propensity score models and multiple covariate‐conditional mean models to be specified. The resulting estimators are consistent provided that any one of these models is correct. An iterative algorithm is provided for implementing this more robust estimator and practical considerations for specifying multiple models are discussed. We evaluate the performance of the proposed method through Monte Carlo simulations and apply the proposed multiply robust estimator to analyze the Botswana Combination Prevention Project, a large HIV prevention CRT designed to evaluate whether a combination of HIV‐prevention measures can reduce HIV incidence.

JRSSB Machine Learning Networks & Graphs · 发表于 2024-02-05

Authors’ reply to the Discussion of ‘From denoising diffusions to denoising Markov models’ at the Discussion Meeting on ‘Probabilistic and statistical aspects of machine learning’

Joe Benton, Yuyang Shi, Valentin De Bortoli, George Deligiannidis, Arnaud Doucet

摘要

We would like to thank all the discussants for their contributions and comments on our work. The question of why and on what class of distributions denoising Markov models (DMMs) work well in practice remains solidly open, as was highlighted by several discussants. Nevertheless, we may offer several hypotheses supported by recent investigations. Firstly, for a denoising diffusion in Rd approximating a data distribution supported on a submanifold, for small times t the score function ∇logqt(x) points locally in the direction towards the manifold and explodes in magnitude. This behaviour is induced by the implicit Kullback-Leibler (KL) objective we minimize and forces the model to sample from close to the manifold, as suggested by Papp, Fearnhead and Sherlock, helping the model to recover the underlying data manifold (Stanczuk et al., 2023; Wenliang & Moran, 2023). We hypothesize that something similar may happen for DMMs more generally, whereby they pick up the geometry of the data distribution through observing local changes in qt(x) for small times. Other work has identified conditions under which diffusion models on Rd will accurately reconstruct the data manifold (Pidstrigach, 2022), but diffusion models in practice operate far from the regime discussed therein. In any case, a generative model which perfectly memorized the empirical distribution would be useless, as it would not generalize beyond the training data. So, to understand the success of DMMs, we must also understand how the inductive bias of the score approximation network allows them to interpolate between observed samples. It would appear that diffusion models have powerful inductive biases encouraging this interpolation, since it has been observed that diffusion models trained with the same architecture on disjoint training datasets often end up learning to generate very similar images (Kadkhodaie et al., 2023). Remarkably, it has ever been observed experimentally by Zhang et al. (2023) that ‘when starting with the same initial noise input and sampling with a deterministic solver, diffusion models tend to produce nearly identical output content. This consistency holds true regardless of the choices of model architectures and training procedures’. Work bounding the error of diffusion models on Rd has decomposed the difference between the data distribution and the learned approximation into three terms, one arising from the approximation of the marginal qT of the forward process at time T with the reference distribution qref⁠, one arising from the time-discretization of the reverse stochastic differential equation (SDE), and one arising from the error of the score approximation (Chen et al., 2023c; Chen et al., 2023a). Empirically, we find that the last of these dominates, suggesting that the observed generalization is coming mostly from the manner in which the score is approximated. There are three factors affecting the approximation learned for the score, which were highlighted in Wang’s response. The first is our choice of the ELBO training objective, which we expect generally imparts a favourable bias—on Rd it reduces to the L2 score matching objective, which has an elegant and intuitive interpretation in terms of denoising score matching with multiple noise levels (Song & Ermon, 2019; Vincent, 2011). The second is the use of gradient-based optimizers such as stochastic gradient descent or Adam to learn the parameters of our approximation, which as Wang highlights often finds solutions which generalize well. The third is the architecture of the neural network used to parameterize the score approximation. A typical choice of architecture is the UNet, which seems empirically to perform well (Ho et al., 2020). There have been some initial investigations into the appropriateness of the inductive bias of UNets for diffusion models [see, for example Williams et al., 2023 or the experiments provided by Kadkhodaie et al. (2023) and Gilliot, Andrieu, Lee, Liu, and Whitehouse]. However, our understanding of the relevance of the neural network architecture is far from complete, and the questions of which biases are induced by the common choices of architectures and how to encode a given set of biases through an appropriate such choice remains understudied. In general, much of the existing literature on the inductive biases of diffusion models applies only to the case of models on Rd⁠. Understanding the bias of arbitrary DMMs will inevitably be a much greater challenge. One of the main practical considerations when implementing DMMs is how to simulate the forward and reverse processes. While we focused mostly on simple first-order methods in our paper, finding improved inference techniques may greatly improve the performance of DMMs in practice, since the number of steps required to simulate the reverse process dictates the number of neural function evaluations required, which is typically the limiting factor in performance. A wide range of techniques have been developed for speeding up diffusion models in Rd⁠, including the probability flow ordinary differential equation (probability flow ODE, or PF ODE) (Song, Sohl-Dickstein, et al., 2021), consistency models (Song et al., 2023), distillation (Salimans & Ho, 2022), using higher order ODE simulation methods (Karras et al., 2022), or using non-Markovian noising processes such as in the denoising diffusion implicit model (DDIM) or flow matching methods (Lipman et al., 2023; Liu et al., 2023; Song, Meng, et al., 2021). Unfortunately, most of these techniques do not immediately transfer to the more general DMM setting. Nevertheless, we are hopeful that specific techniques can speed up inference for particular classes of DMMs. For example, the tau-leaping method developed by Campbell et al. (2022) and which we use in Section 6.2 can be viewed as one such method. Other possibilities may include extending the probability flow ODE to new state spaces—indeed, De Bortoli et al. (2022) show how the PF ODE can be extended to the Riemannian manifold setting—though we are not currently aware of a straightforward and computationally tractable generalization of the PF ODE to arbitrary DMMs. Alternatively, we could find higher order simulation methods or adaptive time-stepping procedures for other classes of Markov processes, as suggested by Wilkinson, which would mitigate the need for time-rescaling. In response to Kent’s question regarding the choice of hyperparameters for the Ornstein-Uhlenbeck (OU) forward process, we note that the forward process should be run until (approximate) convergence—that is, until the information in the original distribution is essentially completely destroyed. Assuming we do this, any choices of λ and σ will in fact lead to an equivalent diffusion model, up to linear rescalings of time and space. Therefore, there is little theoretical difference to be made by tuning the parameters λ and σ, beyond the inductive bias that particular choices impart when learning the score approximation. In practice, it is found that it is easiest to learn the score function if the data are normalized to take values in some standard interval. For example, for data on Rd⁠, we typically rescale the data to lie in the interval [−1,1]⁠, or have identity covariance, and then choose our forward process to have the standard Gaussian as its invariant distribution (Ho et al., 2020). For a similar reason, we also typically apply a time-rescaling function β(t) to make the score easier to learn near t=0⁠, as described in Song, Sohl-Dickstein, et al. (2021). There are several reasons why we tend to prefer parameterizing the reverse process in terms of a logarithmic gradient of β(x,t) rather than via β(x,t) directly. The first is that we expect the ∇log operator to smooth the target substantially. At the optimum, β(x,t) is equal to the marginal qt(x)⁠, which may vary over many orders of magnitude when t is close to 0. Accordingly, logβ(x,t) tends to be a much more stable training target. Second, we find that the objective IISM and the reverse process are often more naturally parameterized by ∇logβ(x,t) or its equivalent, as can be seen for example in the real diffusion case in Appendix J.1. If it is ∇logβ(x,t) which features in both our training objective and our description of the reverse process, we believe that it makes most sense to target this quantity directly. Finally, we find that this choice leads empirically to more stable training, validating our design decision. We expect this to be the case in most (though perhaps not all) applications of DMMs. Concerning our decision not to enforce symmetry constraints on logr(x,y;t) in the continuous-time Markov chain (CTMC) example, this was mostly for practical convenience, since it is simpler to parameterize the unconstrained function. In addition, we did not find that the performance of our models were noticably negatively affected by this lack of symmetry, as Papp, Fearnhead, and Sherlock themselves note. We speculate that this may be a particular instance of a common phenomenon in machine learning whereby discarding certain domain-specific knowledge and optimizing over an unconstrained state space turns out to perform just as well, if not better, than a more heavily customized model. One of our main motivations for introducing DMMs was to make it possible to apply the powerful denoising diffusion modelling technique to a much broader set of problems. While we had limited space to address such applications in our main work, we were extremely gratified by the many potential real-world applications of DMMs that discussants highlighted, including in theoretical neurobiology (Cialfi), learning highly concentrated directional distributions (Kent), and creating synthetic datasets for statistical disclosure control (Jackson). Applications to spaces with mixed discrete and continuous components, as suggested by Wilkinson, are common in protein generation; see e.g. Morehead et al. (2024). We expect DMMs to be most useful for very high-dimensional or multi-modal distributions. The primary purpose of providing the g-and-k example in Section 6.1 was to demonstrate that DMMs can also be competitive in standard Bayesian settings, and to evaluate their performance in a case where the ground truth was known and we could perform a comparison to other methods. We found that DMMs were at least as effective as other simulation-based inference (SBI) methods, though we agree with Nemeth and Wilkinson that the large computational cost of DMMs is a drawback which must be weighed against their benefits when considering whether to apply DMMs in such situations. This computational cost is likely a substantial part of the reason why DMMs are not yet used more widely for SBI. Nemeth notes that the neural networks we use for inference in the g-and-k example are vastly overparameterized compared to both the observed dataset and the underlying number of parameters. However, we tend to view such overparameterization as an asset rather than a hindrance—in the machine learning literature, it is commonly found that such overparameterization provides a favourable inductive bias, leading us to learn a score approximation that generalizes well. Regarding how many parameters are required to have a high degree of statistical accuracy, Chen et al. (2023b) provide bounds on the size of a neural network required for a given accuracy of score approximation in the case where the data distribution is supported on a linear submanifold. Of course, where DMMs really come into their own is on problems that are much more complicated than the simple g-and-k distribution, such as the image inpainting example discussed later. For such high-dimensional problems, classical SBI techniques are infeasible and denoising-based methods really shine. As Mardia highlights, many methods for defining or approximating distributions on SO(3) may run into problems arising from using singular charts for the manifold. We take this opportunity to clarify how our DMMs avoid this problem by respecting the manifold structure of SO(3)⁠. Mardia rightly points out that we use a wrapping procedure to construct the synthetic data distribution on which we test our DMMs, and that this wrapping procedure could cause our test distributions to be non-smooth. However, this simply means that the distributions we test our method by approximating may be singular—it does not reflect a limitation of our DMM method. Conversely, we would like our learning procedure to be smooth and respect the manifold structure of SO(3)⁠, and indeed this was a significant motivation for introducing DMMs. When we apply DMMs on SO(3)⁠, we use a noising process—the Brownian diffusion—which respects the manifold structure, and we parameterize the reverse process in terms of a function {sθi(x,t)}i=13⁠, which is represented as a three-dimensional vector constrained to lie in the relevant tangent plane (for details, see Section 6.3 and Appendix J.5 of the paper). Accordingly, our parameterization of the reverse process is free of singularities and our learning procedure respects the structure and symmetry of SO(3)⁠. Our decision to use M=16 components for the synthetic test distributions was a somewhat arbitrary design choice made to strike a good balance between significant multi-modality and ease of visual representation. The values of σm are drawn randomly with mean 0.1⁠. Regarding how we assess our methods, we chose to compare to the ground truth for inference problems where it was available in order to verify that our method produced sensible predictions. However, in many cases such as image generation, perhaps the most important metrics will concern the perceptual quality of samples. Hence, for these problems we focused on providing a range of samples from our models, and demonstrating that the samples we get appear to be diverse and representative of the true image manifold. To further assess the performance in the absence of a ground truth, we may also consider using metrics such as the maximum mean discrepancy (Gretton et al., 2012) to assess the distance between the distribution of the generated samples and the training data. Power notes that recent work of Montanari (2023) has shown that real diffusion models are equivalent to stochastic localization sampling processes. This correspondence gives new powerful tools for studying diffusion models and their convergence properties—indeed, in separate work we have recently used this result to provide improved convergence bounds for real diffusion models (Benton et al., 2023). Nevertheless, the denoising perspective on diffusion models retains certain advantages. For example, it suggests natural generalizations of diffusion models using non-Markovian forward processes, as in Song, Meng, et al. (2021) or Lipman et al. (2023). In addition, to make sampling from the observation process tractable, Montanari (2023) rely on a result which essentially categorizes the time-reversal of an SDE as another SDE (their Proposition 1.1). In order to apply the observation perspective on a general state space, we would need an equivalent of this result that held for arbitrary Markov processes. Perhaps this may be done, but our DMM work from our Markovian perspective circumvents the need for such a result—and indeed may be viewed from one perspective as deriving a result of this form, since if we assume the reverse process is learned perfectly, then it will be equal to the time-reversal of the forward process, giving a way to identify this time-reversal in the general case. *(abstract via OpenAlex)*

Biometrika Testing & Uncertainty · 发表于 2024-02-04

On the failure of the bootstrap for Chatterjee’s rank correlation

Zhexiao Lin, Fang Han

摘要

While researchers commonly use the bootstrap to quantify the uncertainty of an estimator, it has been noticed that the standard bootstrap, in general, does not work for Chatterjee’s rank correlation. In this paper, we provide proof of this issue under an additional independence assumption, and complement our theory with simulation evidence for general settings. Chatterjee’s rank correlation thus falls into a category of statistics that are asymptotically normal, but bootstrap inconsistent. Valid inferential methods in this case are Chatterjee’s original proposal for testing independence and the analytic asymptotic variance estimator of Lin & Han (2022) for more general purposes. [Received on 5 April 2023. Editorial decision on 10 January 2024]

JRSSB Latent Variables & Mixtures · 发表于 2024-02-03

Semi-parametric tensor factor analysis by iteratively projected singular value decomposition

Elynn Y Chen, Dong Xia, Chencheng Cai, Jianqing Fan

摘要

This paper introduces a general framework of Semi-parametric TEnsor Factor Analysis (STEFA) that focuses on the methodology and theory of low-rank tensor decomposition with auxiliary covariates. Semi-parametric TEnsor Factor Analysis models extend tensor factor models by incorporating auxiliary covariates in the loading matrices. We propose an algorithm of iteratively projected singular value decomposition (IP-SVD) for the semi-parametric estimation. It iteratively projects tensor data onto the linear space spanned by the basis functions of covariates and applies singular value decomposition on matricized tensors over each mode. We establish the convergence rates of the loading matrices and the core tensor factor. The theoretical results only require a sub-exponential noise distribution, which is weaker than the assumption of sub-Gaussian tail of noise in the literature. Compared with the Tucker decomposition, IP-SVD yields more accurate estimators with a faster convergence rate. Besides estimation, we propose several prediction methods with new covariates based on the STEFA model. On both synthetic and real tensor data, we demonstrate the efficacy of the STEFA model and the IP-SVD algorithm on both the estimation and prediction tasks.

SiM Networks & Graphs · 发表于 2024-02-02

Statistical inference on qualitative differences in the magnitude of an effect

Aaron Hudson, Ali Shojaie

摘要

Qualitative interactions occur when a treatment effect or measure of association varies in sign by sub‐population. Of particular interest in many biomedical settings are absence/presence qualitative interactions, which occur when an effect is present in one sub‐population but absent in another. Absence/presence interactions arise in emerging applications in precision medicine, where the objective is to identify a set of predictive biomarkers that have prognostic value for clinical outcomes in some sub‐population but not others. They also arise naturally in gene regulatory network inference, where the goal is to identify differences in networks corresponding to diseased and healthy individuals, or to different subtypes of disease; such differences lead to identification of network‐based biomarkers for diseases. In this paper, we argue that while the absence/presence hypothesis is important, developing a statistical test for this hypothesis is an intractable problem. To overcome this challenge, we approximate the problem in a novel inference framework. In particular, we propose to make inferences about absence/presence interactions by quantifying the relative difference in effect size, reasoning that when the relative difference is large, an absence/presence interaction occurs. The proposed methodology is illustrated through a simulation study as well as an analysis of breast cancer data from the Cancer Genome Atlas.

SiM Spatial & Time Series Bayesian Methods Latent Variables & Mixtures · 发表于 2024-02-02

Modeling variation in mixture effects over space with a Bayesian spatially varying mixture model

Joseph Boyle, Mary H. Ward, James R. Cerhan, Nat Rothman, David C. Wheeler

摘要

Mixture analysis is an emerging statistical tool in epidemiological research that seeks to estimate the health effects associated with mixtures of several exposures. This approach acknowledges that individuals experience many simultaneous exposures and it can estimate the relative importance of components in the mixture. Health effects due to mixtures may vary over space driven by to political, demographic, environmental, or other differences. In such cases, estimating a global mixture effect without accounting for spatial variation would induce bias in effect estimates and potentially lower statistical power. To date, no methods have been developed to estimate spatially varying chemical mixture effects. We developed a Bayesian spatially varying mixture model that estimates spatially varying mixture effects and the importance weights of components in the mixture, while adjusting for covariates. We demonstrate the efficacy of the model through a simulation study that varies the number of mixtures (one and two) and spatial pattern (global, one‐dimensional, radial) and magnitude of mixture effects, showing that the model is able to accurately reproduce the spatial pattern of mixture effects across a diverse set of scenarios. Finally, we apply our model to a multi‐center case‐control study of non‐Hodgkin lymphoma (NHL) in Detroit, Iowa, Los Angeles, and Seattle. We identify significant spatially varying positive and inverse associations with NHL for two mixtures of pesticides in Iowa and do not find strong spatial effects at the other three centers. In conclusion, the Bayesian spatially varying mixture model represents a novel method for modeling spatial variation in mixture effects.

JRSSB Bayesian Methods · 发表于 2024-02-02

Martingale posterior distributions

Edwin Fong, Chris Holmes, Stephen G Walker

摘要

The prior distribution is the usual starting point for Bayesian uncertainty. In this paper, we present a different perspective that focuses on missing observations as the source of statistical uncertainty, with the parameter of interest being known precisely given the entire population. We argue that the foundation of Bayesian inference is to assign a distribution on missing observations conditional on what has been observed. In the i.i.d. setting with an observed sample of size n, the Bayesian would thus assign a predictive distribution on the missing Yn+1:∞ conditional on Y1:n, which then induces a distribution on the parameter. We utilize Doob’s theorem, which relies on martingales, to show that choosing the Bayesian predictive distribution returns the conventional posterior as the distribution of the parameter. Taking this as our cue, we relax the predictive machine, avoiding the need for the predictive to be derived solely from the usual prior to posterior to predictive density formula. We introduce the martingale posterior distribution, which returns Bayesian uncertainty on any statistic via the direct specification of the joint predictive. To that end, we introduce new predictive methodologies for multivariate density estimation, regression and classification that build upon recent work on bivariate copulas.

JASA Bayesian Methods Latent Variables & Mixtures Spatial & Time Series · 发表于 2024-02-02

Fitting Latent Non-Gaussian Models Using Variational Bayes and Laplace Approximations

Rafael Cabral, David Bolin, Håvard Rue

摘要

Latent Gaussian models (LGMs) are perhaps the most commonly used class of models in statistical applications. Nevertheless, in areas ranging from longitudinal studies in biostatistics to geostatistics, it is easy to find datasets that contain inherently non-Gaussian features, such as sudden jumps or spikes, that adversely affect the inferences and predictions made using an LGM. These datasets require more general latent non-Gaussian models (LnGMs) that can handle automatically these non-Gaussian features. However, fast implementation and easy-to-use software are lacking, preventing the broad applicability of LnGMs. In this article, we derive variational Bayes algorithms for fast and scalable inference of LnGMs. The approximation leads to an LGM that downweights extreme events in the latent process, reducing their influence and leading to more robust inferences. It can be applied to a wide range of models, such as autoregressive processes for time series, simultaneous autoregressive models for areal data, and spatial Matérn models. To facilitate Bayesian inference, we introduce the ngvb package, where LGMs implemented in R-INLA can be easily extended to LnGMs by adding a single line of code. Supplementary materials for this article are available online. *(abstract via OpenAlex)*

JRSSB Bayesian Methods Testing & Uncertainty · 发表于 2024-02-02

Authors' reply to the Discussion of ‘Martingale Posterior Distributions’

Edwin Fong, Chris Holmes, Stephen G Walker

摘要

We thank all the discussants for their contributions which highlight important aspects of our work and additionally provide many hints for future directions of research. We are pleased that the discussions were generally receptive to expanding beyond the traditional likelihood-prior framework of Bayes, and to shift the focus to the predictive distribution. We begin our rejoinder by summarizing some core themes that have been raised by the discussants, followed by more detailed discussions. These are as follows: Some discussants highlighted the role of the already observed data, y1:n⁠. In our view, given the observed information, the predictive p(yn+1|y1:n) is the analysts’ best density estimator for the next observation, and this will reflect the underlying structure of the data. There is no explicit need to model the observed y1:n⁠. From an intuitive perspective, there is no uncertainty in what has already been seen, i.e. the data y1:n are fully known and we treat it as such. Hence, assigning a probability model to what has been seen could be argued as unnecessary. The martingale posterior is not to be regarded as a replacement of traditional Bayesian analysis. If it is deemed that the formal Bayesian model, derived from a prior to posterior, leads to the best predictive, then this remains a part of the framework espoused in the article. We argue however that there are settings where this rigid framework is questionable, such as when tasked with eliciting objective priors in the absence of substantive prior information. There is much discussion on the formal structure of the data to be studied. For assumed i.i.d. data, the Bayesian adopts an exchangeable structure from their symmetry of beliefs, not because they actually believe the measured states are physically dependent, but because the predictives, i.e. density estimators for the next observation, evolve as more is seen. It is the predictive which depends on the past, rather than the variable coming from it. For example, when considering a sequence of tosses of a coin, the outcome of one toss does not influence the outcome of any other toss, but an observed toss is certainly informative for estimating the probability of getting heads in the next toss. Exchangeability can be viewed as providing a learning model, as can a c.i.d. structure for Yn+1:∞⁠, which we adopt as the basic framework for the missing data. The role of the observations y1:n in determining the initial predictive Pn was a common theme in multiple comments. As highlighted by Dawid, the statistical model is the sequence of predictives. While we agree that it is important to link the observed y1:n to the unobserved Yn+1:∞, we argue that the traditional starting point of an assumed structure for y1:n can now play a secondary role to our primary goal, which is to construct the best predictive Pn for Yn+1 given y1:n⁠. In our view, y1:n has already been observed and there is no longer any uncertainty in their values—only uncertainty in Yn+1:∞⁠. Draper and Guo highlight the asymptotic equivalence between frequentist and Bayesian bootstraps, which could be a useful approximation in some instances. However, we must reiterate that the source of uncertainty is entirely different: the frequentist considers y1:n as random, whereas the Bayesian considers Yn+1:∞ as random given y1:n⁠. The elicitation of the distribution of Yn+1:∞ given y1:n is quite a general problem, of which there are many approaches. A sequential approach would entail a sequence of one-step ahead predictive distributions. Indeed, this becomes the statistical problem. For example, Dawid’s prequential approach, where one specifies P0,P1,…,Pn−1 and assesses its fit to the observed data y1:n is a natural approach, and we indeed carry this out to select hyperparameters and learn Pn through our copula updates. The secondary focus on the data structure y1:n does not preclude us from using knowledge of the data generating mechanism. In the presence of a statistical model, e.g. in the scenario proposed by Clarke, we are free to use the plug-in predictive p(⋅|y1:n)=f(⋅|θn^)⁠, with (θ^m)m>n as the possible martingale, which was recently proposed by Holmes and Walker (2023). This predictive can be based on a more conventional statistical model and estimator, such as the maximum likelihood estimator. One can then view this as setting Pn to the best density estimate of the data generating distribution given current knowledge. Of course, if we know something about the structure of Yn+1:∞⁠, this may help us elicit the best predictive. One is also free to incorporate prior knowledge in addition to the observations. For example, Rigo and Antoniano-Villalobos highlight that given a single observation y1, the empirical measure as the predictive would return us no uncertainty in the martingale posterior. However, we argue that in this setting, it would be natural to incorporate prior information (such as historical data) or smoothing as suggested by Antoniano-Villalobos instead of relying on the empirical distribution for a single datum. Note that all bootstraps fail with a single observation. When n is large, and in the absence of substantive background knowledge, having to specify a prior will often be a distraction. The martingale posterior frees the Bayesian from this restriction. When faced with the issue of selecting a predictive, we agree with the discussants that the standard battery of model selection tools can be utilized, such as prequential tests and scores as suggested by Dawid and Huk et al. This can also be applied to select hyperparameters such as the bandwidth ρ or initial guess p0(y)⁠, which has close connections to empirical Bayes as highlighted by Gu and Yin. We agree with Swallow and Gu and Yin that uncertainty in model selection or variable selection is one of the strengths of the Bayesian framework. Future directions of research would be to investigate what the missing future data and the decision problem is under the martingale posterior framework. To date, given a model, we are able to assess the uncertainty via a martingale posterior distribution for the object of interest. We are currently working on decision problems where the idea is to make a choice with associated uncertainty quantification. The idea here is to generate Yn+1:∞ which makes the decision known, whereas each different sequence of future data could present a different decision. We agree with Ray and Szabo and Wijayatunga on the importance of frequentist consistency of the predictive, which is also connected to the issue of model selection. Ray and Szabo provide an interesting example of posterior inconsistency, which we believe can be alleviated through the above discussion on model selection. If it were known the model was either normal or Cauchy, there are a number of ways this could be determined from y1:n and the appropriate predictive selected. An interesting future direction would be to investigate how good frequentist properties for the predictive impact the posterior. Lauritzen, Ascolani et al., Rossell, and Mena highlighted that it may sometimes be easier to elicit a likelihood and prior. We agree with this sentiment, and reiterate that the martingale posterior is not aiming to replace Bayes, as it encompasses it. We envision many scenarios, as those suggested by the discussants, where a prior to posterior calculation leads to the best predictive. However, there may be situations in which we do not have any prior information, such as those considered by objective Bayesians. One of the goals of the article is to highlight that the prior distribution is not needed for Bayesian inference, where the uncertainty arises from the unseen Yn+1:∞⁠. Aside from the Bayesian bootstrap, another illuminating example is the plug-in predictive as discussed in the previous section and Holmes and Walker (2023). Here, our predictive f(⋅|θn^) is based on a statistical model, so we have discarded the prior distribution component of the traditional Bayesian setup. There is now no acknowledged uncertainty in the value of θ^n from a plug-in density estimator, but statistical uncertainty in θ∞ is constructed via the sampling from the sequence Yn+1:∞⁠. We thus do not entirely agree with Rossell’s comment that we are not prior-free, as there is no explicit prior distribution on θ. However, if you have prior information, e.g. in the form an initial predictive density p0⁠, then this can also be evaluated in a data-driven way. Another setting where eliciting a prior could be deemed inconvenient is in Bayesian non-parametrics, where priors can be technical and not necessarily intuitive. As highlighted by Ray and Szabo, predictive distributions elicited through the likelihood-prior construction may be misleading due to large prior null sets, especially in the non-parametric setting. Eliciting and evaluating the predictive directly could be a way to bypass these issues, though other new challenges arise, and we look forward to exploring this direction further. Another reason to bypass the likelihood-prior construction is due to computational reasons. Depending on the setting, computation with predictives can be noticeably faster than traditional posterior sampling, e.g. the Bayesian bootstrap or predictive resampling as demonstrated by Moya, but we acknowledge that this is not always universal. Rossell highlights that optimization can be costly, but we agree with Gu and Yin and Moya that parallelization is easily carried out and is less straightforward with MCMC methods. Furthermore, predictive resampling as a computational algorithm is still in its infancy. We hope that additional computational methods or approximations can be developed under the martingale posterior framework. Another common theme among discussants is the structure of the imputed data Yn+1:∞⁠, and going beyond exchangeability. Ascolani et al. and Catalano et al. discuss whether one can identify novel direct predictive updates which are exchangeable (perhaps only finitely), and Mena and Antoniano-Villalobos questions the importance of exchangeability. While we do indeed average over permutations in our construction of Pn to ensure some permutation invariance of the predictive, this is mostly motivated by practical concerns. It may be undesirable for the analysis to be highly sensitive to the order in which we process the data, but we do not view exchangeability as crucial for the martingale posterior. Rigo highlights the importance of the Ionescu-Tulcea theorem in our setting, with which we agree. Indeed, when we are only concerned with the structure of Yn+1:∞⁠, our choice of c.i.d. sequences gives us sufficient asymptotic structure for the i.i.d. setting, but as seen in the plug-in examples above, this is not the only way to guarantee convergence or coherence under the i.i.d. setting. Closely connected to the above is the question on the extension of our framework beyond the i.i.d. setting. Lauritzen questions if the martingale posterior framework is sufficiently general, and provides a pure birth process example. Similarly, Dawid asks what is necessary under a Markov setting. AntonianoVillalobos also highlights the importance of assumptions such as smoothness and sparsity when our object of interest requires more than a countable number of observations. While the generality of the martingale posterior still requires investigation, we provide some preliminary hints below. For example, if the data are Markov AR(1) as in Dawid and Mena’s example, the datum yn would play a significant role in predicting yn+1⁠, and the predictive would be of the form p(yn+1|θ^n,yn)⁠. The corresponding structure for Yn+1:∞ may not be exactly Markov but the outcome would be that it is asymptotically Markov as θN→θ∞ eventually. This line of thinking may also allow us to generalize beyond the martingale condition as asked by Mena. Hence, the structure is to produce the best form of predictive which would arise from the structure of the likelihood function, and a formal characterization of this would be of great interest. The model proposed by Lauritzen can also be handled using the martingale posterior framework. Suppose we have observed X(t) up to time T from which we have λb⁠. We now take the predictive to be, for any h>0⁠, and t≥T⁠, and We can discretize this in an obvious way: we use intervals of time h/Y[N−1]⁠, for N=1,2,… with Y[0]=X[T]⁠, for some small h. Hence, jumps can occur at times t(N)=t(N−1)+h/Y[N−1]⁠, with t(0)=T⁠, and where the first term on the right is S(T) and the final term on the right is Nh. As we predictively resample Y(N)⁠, we update λb as we go. Suppose at iteration N we have Y(N) and λ^N⁠, then It is easy to see that (λ^N)N≥0 is a martingale. We generated a process using λ=1.5 and observe X(T)=15 with T=1.633 and S(T)=10 , with λb=1.4⁠. We then sample the martingale posterior for N=1,…,10000⁠, which is far more than is required for the martingale to converge. Repeating this gives us 1,000 samples from the martingale posterior, as shown in Figure 1, where the mean of the samples is 1.40. Lauritzen’s second example with both {λ,w} seems more challenging, and we defer this to future work. Histogram representation of samples from the martingale posterior for λ. In the above two examples, we have relied on the parametric statistical model to ensure structure in Yn+1:∞⁠, but without the reliance on the full Bayesian machinery (i.e. no prior distribution). An extension to hierarchical models would be interesting, as highlighted by Ascolani et al., and it would be of interest if the structures considered above can be incorporated directly into the predictive. We believe this to be a fruitful direction of future research. *(abstract via OpenAlex)*

JASA Spatial & Time Series · 发表于 2024-02-02

A Composite Likelihood-Based Approach for Change-Point Detection in Spatio-Temporal Processes

Zifeng Zhao, Ting Fung Ma, Wai Leong Ng, Chun Yip Yau

摘要

This article develops a unified and computationally efficient method for change-point estimation along the time dimension in a nonstationary spatio-temporal process. By modeling a nonstationary spatio-temporal process as a piecewise stationary spatio-temporal process, we consider simultaneous estimation of the number and locations of change-points, and model parameters in each segment. A composite likelihood-based criterion is developed for change-point and parameter estimation. Under the framework of increasing domain asymptotics, theoretical results including consistency and distribution of the estimators are derived under mild conditions. In contrast to classical results in fixed dimensional time series that the localization error of change-point estimator is Op(1), exact recovery of true change-points is possible in the spatio-temporal setting. More surprisingly, the consistency of change-point estimation can be achieved without any penalty term in the criterion function. In addition, we further establish consistency of the change-point estimator under the infill asymptotics framework where the time domain is increasing while the spatial sampling domain is fixed. A computationally efficient pruned dynamic programming algorithm is developed for the challenging criterion optimization problem. Extensive simulation studies and an application to the U.S. precipitation data are provided to demonstrate the effectiveness and practicality of the proposed method. Supplementary materials for this article are available online. *(abstract via OpenAlex)*

AoS Optimization & Learning Theory Meta-Analysis & Data Integration · 发表于 2024-02-01

Transfer learning for contextual multi-armed bandits

Changxiao Cai, T. Tony Cai, Hongzhe Li

摘要

Motivated by a range of applications, we study in this paper the problem of transfer learning for nonparametric contextual multi-armed bandits under the covariate shift model, where we have data collected from source bandits before the start of the target bandit learning. The minimax rate of convergence for the cumulative regret is established and a novel transfer learning algorithm that attains the minimax regret is proposed. The results quantify the contribution of the data from the source domains for learning in the target domain in the context of nonparametric contextual multi-armed bandits. In view of the general impossibility of adaptation to unknown smoothness, we develop a data-driven algorithm that achieves near-optimal statistical guarantees (up to a logarithmic factor) while automatically adapting to the unknown parameters over a large collection of parameter spaces under an additional self-similarity assumption. A simulation study is carried out to illustrate the benefits of utilizing the data from the source domains for learning in the target domain.

AoS Other · 发表于 2024-02-01

The right complexity measure in locally private estimation: It is not the Fisher information

John C. Duchi, Feng Ruan

摘要

We identify fundamental tradeoffs between statistical utility and privacy under local models of privacy in which data is kept private even from the statistician, providing instance-specific bounds for private estimation and learning problems by developing the local minimax risk. In contrast to approaches based on worst-case (minimax) error, which are conservative, this allows us to evaluate the difficulty of individual problem instances and delineate the possibilities for adaptation in private estimation and inference. Our main results show that the local modulus of continuity of the estimand with respect to the variation distance—as opposed to the Hellinger distance central to classical statistics—characterizes rates of convergence under locally private estimation for many notions of privacy, including differential privacy and its relaxations. As consequences of these results, we identify an alternative to the Fisher information for private estimation, giving a more nuanced understanding of the challenges of adaptivity and optimality.

AoS Nonparametric & Semiparametric High-Dimensional · 发表于 2024-02-01

Testing for independence in high dimensions based on empirical copulas

Axel Bücher, Cambyse Pakzad

摘要

Testing for pairwise independence for the case where the number of variables may be of the same size or even larger than the sample size has received increasing attention in the recent years. We contribute to this branch of the literature by considering tests that allow to detect higher-order dependencies. The proposed methods are based on connecting the problem to copulas and making use of the Moebius transformation of the empirical copula process; an approach that is related to Lancaster interactions and that has already been used successfully for the case where the number of variables is fixed. Based on a martingale central limit theorem, it is shown that respective test statistics converge to the standard normal distribution, allowing for straightforward definition of critical values. The results are illustrated by a Monte Carlo simulation study.

AoS Optimization & Learning Theory · 发表于 2024-02-01

Supervised homogeneity fusion: A combinatorial approach

Wen Wang, Shihao Wu, Ziwei Zhu, Ling Zhou, Peter X.-K. Song

摘要

Fusing regression coefficients into homogeneous groups can unveil those coefficients that share a common value within each group. Such groupwise homogeneity reduces the intrinsic dimension of the parameter space and unleashes sharper statistical accuracy. We propose and investigate a new combinatorial grouping approach called L0-Fusion that is amenable to mixed integer optimization (MIO). On the statistical aspect, we identify a fundamental quantity called MSE grouping sensitivity that underpins the difficulty of recovering the true groups. We show that L0-Fusion achieves grouping consistency under the weakest possible requirement of the grouping sensitivity: if this requirement is violated, then the minimax risk of group misspecification will fail to converge to zero. Moreover, we show that in the high-dimensional regime, one can apply L0-Fusion with a sure screening set of features without any essential loss of statistical efficiency, while reducing the computational cost substantially. On the algorithmic aspect, we provide an MIO formulation for L0-Fusion along with a warm start strategy. Simulation and real data analysis demonstrate that L0-Fusion exhibits superiority over its competitors in terms of grouping accuracy.

AoS Econometrics & Economics Clinical Trials · 发表于 2024-02-01

Statistical-computational trade-offs in tensor PCA and related problems via communication complexity

Rishabh Dudeja, Daniel Hsu

摘要

Tensor PCA is a stylized statistical inference problem introduced by Montanari and Richard to study the computational difficulty of estimating an unknown parameter from higher-order moment tensors. Unlike its matrix counterpart, Tensor PCA exhibits a statistical-computational gap, that is, a sample size regime where the problem is information-theoretically solvable but conjectured to be computationally hard. This paper derives computational lower bounds on the run-time of memory bounded algorithms for Tensor PCA using communication complexity. These lower bounds specify a trade-off among the number of passes through the data sample, the sample size and the memory required by any algorithm that successfully solves Tensor PCA. While the lower bounds do not rule out polynomial-time algorithms, they do imply that many commonly-used algorithms, such as gradient descent and power method, must have a higher iteration count when the sample size is not large enough. Similar lower bounds are obtained for non-Gaussian component analysis, a family of statistical estimation problems in which low-order moment tensors carry no information about the unknown parameter. Finally, stronger lower bounds are obtained for an asymmetric variant of Tensor PCA and related statistical estimation problems. These results explain why many estimators for these problems use a memory state that is significantly larger than the effective dimensionality of the parameter of interest.

AoS High-Dimensional Testing & Uncertainty Networks & Graphs · 发表于 2024-02-01

StarTrek: Combinatorial variable selection with false discovery rate control

Lu Zhang, Junwei Lu

摘要

Variable selection on the large-scale networks has been extensively studied in the literature. While most of the existing methods are limited to the local functionals especially the graph edges, this paper focuses on selecting the discrete hub structures of the networks. Specifically, we propose an inferential method, called StarTrek filter, to select the hub nodes with degrees larger than a certain thresholding level in the high-dimensional graphical models and control the false discovery rate (FDR). Discovering hub nodes in the networks is challenging: there is no straightforward statistic for testing the degree of a node due to the combinatorial structures; complicated dependence in the multiple testing problem is hard to characterize and control. In methodology, the StarTrek filter overcomes this by constructing p-values based on the maximum test statistics via the Gaussian multiplier bootstrap. In theory, we show that the StarTrek filter can control the FDR by providing accurate bounds on the approximation errors of the quantile estimation and addressing the dependence structures among the maximal statistics. To this end, we establish novel Cramér-type comparison bounds for the high-dimensional Gaussian random vectors. Compared to the Gaussian comparison bound via the Kolmogorov distance established by Chernozhukov, Chetverikov and Kato (Ann. Statist. 42 (2014) 1787–1818), our Cramér-type comparison bounds establish the relative difference between the distribution functions of two high-dimensional Gaussian random vectors, which is essential in the theoretical analysis of FDR control. Moreover, the StarTrek filter can be applied to general statistical models for FDR control of discovering discrete structures such as simultaneously testing the sparsity levels of multiple high-dimensional linear models. We illustrate the validity of the StarTrek filter in a series of numerical experiments and apply it to the genotype-tissue expression dataset to discover central regulator genes.

AoS Machine Learning Optimization & Learning Theory · 发表于 2024-02-01

Settling the sample complexity of model-based offline reinforcement learning

Gen Li, Laixi Shi, Yuxin Chen, Yuejie Chi, Yuting Wei

摘要

This paper is concerned with offline reinforcement learning (RL), which learns using precollected data without further exploration. Effective offline RL would be able to accommodate distribution shift and limited data coverage. However, prior results either suffer from suboptimal sample complexities or incur high burn-in cost to reach sample optimality, thus posing an impediment to efficient offline RL in sample-starved applications. We demonstrate that the model-based (or “plug-in”) approach achieves minimax-optimal sample complexity without any burn-in cost for tabular Markov decision processes (MDPs). Concretely, consider a γ-discounted infinite-horizon (resp., finite-horizon) MDP with S states and effective horizon 11−γ (resp., horizon H), and suppose the distribution shift of data is reflected by some single-policy clipped concentrability coefficient Cclipped⋆. We prove that model-based offline RL yields ε-accuracy with a sample complexity of SCclipped⋆ (1−γ)3ε2(infinite-horizon MDPs),H4SCclipped⋆ ε2(finite-horizon MDPs), up to log factor, which is minimax optimal for the entire ε-range. The proposed algorithms are “pessimistic” variants of value iteration with Bernstein-style penalties, and do not require sophisticated variance reduction. Our analysis framework is established upon delicate leave-one-out decoupling arguments in conjunction with careful self-bounding techniques tailored to MDPs.

AoS High-Dimensional · 发表于 2024-02-01

Rates of estimation for high-dimensional multireference alignment

Zehao Dou, Zhou Fan, Harrison H. Zhou

摘要

We study the continuous multireference alignment model of estimating a periodic function on the circle from noisy and circularly-rotated observations. Motivated by analogous high-dimensional problems that arise in cryo-electron microscopy, we establish minimax rates for estimating generic signals that are explicit in the dimension K. In a high-noise regime with noise variance σ2≳K, for signals with Fourier coefficients of roughly uniform magnitude, the rate scales as σ6 and has no further dependence on the dimension. This rate is achieved by a bispectrum inversion procedure, and our analyses provide new stability bounds for bispectrum inversion that may be of independent interest. In a low-noise regime where σ2≲K/logK, the rate scales instead as Kσ2, and we establish this rate by a sharp analysis of the maximum likelihood estimator that marginalizes over latent rotations. A complementary lower bound that interpolates between these two regimes is obtained using Assouad’s hypercube lemma. We extend these analyses also to signals whose Fourier coefficients have a slow power law decay.

AoS High-Dimensional Nonparametric & Semiparametric · 发表于 2024-02-01

Rank-based indices for testing independence between two high-dimensional vectors

Yeqing Zhou, Kai Xu, Liping Zhu, Runze Li

摘要

To test independence between two high-dimensional random vectors, we propose three tests based on the rank-based indices derived from Hoeffding’s D, Blum–Kiefer–Rosenblatt’s R and Bergsma–Dassios–Yanagimoto’s τ∗. Under the null hypothesis of independence, we show that the distributions of the proposed test statistics converge to normal ones if the dimensions diverge arbitrarily with the sample size. We further derive an explicit rate of convergence. Thanks to the monotone transformation-invariant property, these distribution-free tests can be readily used to generally distributed random vectors including heavily-tailed ones. We further study the local power of the proposed tests and compare their relative efficiencies with two classic distance covariance/correlation based tests in high-dimensional settings. We establish explicit relationships between D, R, τ∗ and Pearson’s correlation for bivariate normal random variables. The relationships serve as a basis for power comparison. Our theoretical results show that under a Gaussian equicorrelation alternative: (i) the proposed tests are superior to the two classic distance covariance/correlation based tests if the components of random vectors have very different scales; (ii) the asymptotic efficiency of the proposed tests based on D, τ∗ and R are sorted in a descending order.

AoS Spatial & Time Series Latent Variables & Mixtures Econometrics & Economics · 发表于 2024-02-01

Rank and factor loadings estimation in time series tensor factor model by pre-averaging

Weilin Chen, Clifford Lam

摘要

The idiosyncratic components of a tensor time series factor model can exhibit serial correlations, (e.g., finance or economic data), ruling out many state-of-the-art methods that assume white/independent idiosyncratic components. While the traditional higher order orthogonal iteration (HOOI) is proved to be convergent to a set of factor loading matrices, the closeness of them to the true underlying factor loading matrices are in general not established, or only under i.i.d. Gaussian noises. Under the presence of serial and cross-correlations in the idiosyncratic components and time series variables with only bounded fourth-order moments, for tensor time series data with tensor order two or above, we propose a pre-averaging procedure that can be considered a random projection method. The estimated directions corresponding to the strongest factors are then used for projecting the data for a potentially improved re-estimation of the factor loading spaces themselves, with theoretical guarantees and rate of convergence spelt out when not all factors are pervasive. We also propose a new rank estimation method, which utilizes correlation information from the projected data. Extensive simulations are performed and compared to other state-of-the-art or traditional alternatives. A set of tensor-valued NYC taxi data is also analyzed.

AoS Other · 发表于 2024-02-01

Maximum likelihood for high-noise group orbit estimation and single-particle cryo-EM

Zhou Fan, Roy R. Lederman, Yi Sun, Tianhao Wang, Sheng Xu

摘要

Motivated by applications to single-particle cryo-electron microscopy (cryo-EM), we study several problems of function estimation in a high noise regime, where samples are observed after random rotation and possible linear projection of the function domain. We describe a stratification of the Fisher information eigenvalues according to transcendence degrees of graded pieces of the algebra of group invariants, and we relate critical points of the log-likelihood landscape to a sequence of moment optimization problems, extending previous results for a discrete rotation group without projections. We then compute the transcendence degrees and forms of these optimization problems for several examples of function estimation under SO(2) and SO(3) rotations, including a simplified model of cryo-EM as introduced by Bandeira, Blum-Smith, Kileel, Niles-Weed, Perry and Wein. We affirmatively resolve conjectures that third-order moments are sufficient to locally identify a generic signal up to its rotational orbit in these examples. For low-dimensional approximations of the electric potential maps of two small protein molecules, we empirically verify that the noise scalings of the Fisher information eigenvalues conform with our theoretical predictions over a range of SNR, in a model of SO(3) rotations without projections.

AoS Other · 发表于 2024-02-01

Estimation and inference for minimizer and minimum of convex functions: Optimality, adaptivity and uncertainty principles

T. Tony Cai, Ran Chen, Yuancheng Zhu

摘要

Optimal estimation and inference for both the minimizer and minimum of a convex regression function under the white noise and nonparametric regression models are studied in a nonasymptotic local minimax framework, where the performance of a procedure is evaluated at individual functions. Fully adaptive and computationally efficient algorithms are proposed and sharp minimax lower bounds are given for both the estimation accuracy and expected length of confidence intervals for the minimizer and minimum. The nonasymptotic local minimax framework brings out new phenomena in simultaneous estimation and inference for the minimizer and minimum. We establish a novel uncertainty principle that provides a fundamental limit on how well the minimizer and minimum can be estimated simultaneously for any convex regression function. A similar result holds for the expected length of the confidence intervals for the minimizer and minimum.

AoS Causal Inference Networks & Graphs Latent Variables & Mixtures · 发表于 2024-02-01

Characterization of causal ancestral graphs for time series with latent confounders

Andreas Gerhardus

摘要

In this paper, we introduce a novel class of graphical models for representing time-lag specific causal relationships and independencies of multivariate time series with unobserved confounders. We completely characterize these graphs and show that they constitute proper subsets of the currently employed model classes. As we show, from the novel graphs one can thus draw stronger causal inferences—without additional assumptions. We further introduce a graphical representation of Markov equivalence classes of the novel graphs. This graphical representation contains more causal knowledge than what current state-of-the-art causal discovery algorithms learn. *(abstract via OpenAlex)*

Biometrika Other · 发表于 2024-02-01

Asymptotically constant risk estimator of the time-average variance constant

K W Chan, C Y Yau

摘要

Summary: Estimation of the time-average variance constant is important for statistical analyses involving dependent data. This problem is difficult as it relies on a bandwidth parameter. Specifically, the optimal choices of the bandwidths of all existing estimators depend on the estimand itself and another unknown parameter that is very difficult to estimate. Thus, optimal variance estimation is unachievable. In this paper, we introduce a concept of converging flat-top kernels for constructing variance estimators whose optimal bandwidths are free of unknown parameters asymptotically and hence can be computed easily. We prove that the new estimator has an asymptotically constant risk and is locally asymptotically minimax.

AoS Bayesian Methods · 发表于 2024-02-01

Adaptive variational Bayes: Optimality, computation and applications

Ilsang Ohn, Lizhen Lin

摘要

In this paper, we explore adaptive inference based on variational Bayes. Although several studies have been conducted to analyze the contraction properties of variational posteriors, there is still a lack of a general and computationally tractable variational Bayes method that performs adaptive inference. To fill this gap, we propose a novel adaptive variational Bayes framework, which can operate on a collection of models. The proposed framework first computes a variational posterior over each individual model separately and then combines them with certain weights to produce a variational posterior over the entire model. It turns out that this combined variational posterior is the closest member to the posterior over the entire model in a predefined family of approximating distributions. We show that the adaptive variational Bayes attains optimal contraction rates adaptively under very general conditions. We also provide a methodology to maintain the tractability and adaptive optimality of the adaptive variational Bayes even in the presence of an enormous number of individual models, such as sparse models. We apply the general results to several examples, including deep learning and sparse factor models, and derive new and adaptive inference results. In addition, we characterize an implicit regularization effect of variational Bayes and show that the adaptive variational posterior can utilize this.

AoS Testing & Uncertainty · 发表于 2024-02-01

Adaptive novelty detection with false discovery rate guarantee

Ariane Marandon, Lihua Lei, David Mary, Etienne Roquain

摘要

This paper studies the semisupervised novelty detection problem where a set of “typical” measurements is available to the researcher. Motivated by recent advances in multiple testing and conformal inference, we propose AdaDetect, a flexible method that is able to wrap around any probabilistic classification algorithm and control the false discovery rate (FDR) on detected novelties in finite samples without any distributional assumption other than exchangeability. In contrast to classical FDR-controlling procedures that are often committed to a pre-specified p-value function, AdaDetect learns the transformation in a data-adaptive manner to focus the power on the directions that distinguish between inliers and outliers. Inspired by the multiple testing literature, we further propose variants of AdaDetect that are adaptive to the proportion of nulls while maintaining the finite-sample FDR control. The methods are illustrated on synthetic datasets and real-world datasets, including an application in astrophysics.

SMMR Longitudinal Data · 发表于 2024-02-01

Accounting for informative observation process in transition models of binary longitudinal outcome: Application to medical record data

Joe Bible, Madeleine St. Ville, Paul S Albert, Danping Liu

摘要

When extracting medical record data to form a retrospective cohort, investigators typically focus on a pre-specified study window, and select subjects who had hospital visits during that study window. However, such data extraction may suffer from an informative observation process, since sicker patients may have hospital visits more frequently. For example, Consecutive Pregnancy Study is a retrospective cohort study of women with multiple pregnancies in 23 Utah hospitals from 2003 to 2010, where the interest is to understand the risk factors of recurrent pregnancy outcomes, such as preterm birth. The observation process is informative in the sense that, women with adverse pregnancy outcomes may be less likely/willing/able to endure subsequent pregnancies. We proposed a three-part joint model with shared random effects structure to address this analytic complication. Particularly, a first-order transition model is used to model the longitudinal binary outcome; a gamma regression model is assumed for the inter-pregnancy intervals; a continuation ratio model specifies the probability of continuing with more births in the future. We note that the latter two parts give rise to a parametric cure-rate survival model. The performance of the proposed method was examined in extensive simulation studies, with both correctly and mis-specified models. The analyses of Consecutive Pregnancy Study data further demonstrate the inadequacies of fitting the transition model alone ignoring the informative observation process.

JASA Bayesian Methods · 发表于 2024-01-31

Statistical Modeling with R: A Dual Frequentist and Bayesian Approach for Life Scientists

Christian P. Robert

摘要

Statistical Modeling with R, subtitled A Dual Frequentist and Bayesian Approach for Life Scientists, is a recent book written by Pablo Inchausti. It is composed in a highly personal and congenial s... *(abstract via OpenAlex)*

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 页