2023 年的期刊文章

共 1511 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2023-12-29 到 2023-12-21。

按期刊看(展开)
按主题看(展开)

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 页

JRSSB Machine Learning Clinical Trials Spatial & Time Series 讨论 / 评论 · 发表于 2023-12-29

Ivor Cribben and Anastasiou Andreas’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Andreas Anastasiou, Ivor Cribben

摘要

We extend our congratulations to the authors for their innovative contribution. Here, we address three key points: change-point labelling, handling imbalanced datasets, and computational complexity. Change-point labelling in datasets has many flaws and challenges. The primary flaw is the subjectivity inherent in the task as it often relies on human judgement. This subjectivity introduces biases and inconsistencies. One challenge is the lack of a universally accepted standard for change-point labelling unlike other machine learning classification problems. Hence, it is difficult to compare results across studies, hindering reproducibility and reliability. A further issue is that change-point labelling can be a time-consuming and labour-intensive task, especially for large and complex time series datasets. This process often requires domain expertise and can be impractical for real-time or high-frequency data analysis. Imbalanced class distributions in datasets are another issue. Change-points are often rarer than normal instances, but imbalanced datasets can lead to skewed evaluation results, with methods prioritizing the majority class and failing to effectively detect true change-points or managing an excessive rate of false positives. We wonder whether the authors explored examples of imbalanced data, specifically those involving significant changes in approximately half of the dataset (N/2). It is important to underscore that addressing the labelling and imbalance challenges is pivotal for change-point methods that rely on training neural networks. The sample size used for training neural networks plays a crucial role in determining the model’s performance and generalizability. An excessively large sample size might lead to increased computational costs and training time without significant gains in performance after a certain point. In the first step of the proposed algorithm, there is the necessity of training a neural network using a considerable sample size. A discussion on this topic could convince the reader that the proposed method can be extended to an online framework (as discussed in Section 7). Apart from accuracy, and especially in high-frequency data, online change-point detection methods have to be very computationally efficient in order for users to act promptly. The computational complexity in the simple univariate setting is also crucial to understand extensions to practically meaningful adaptations of the algorithm to multivariate, possibly high-dimensional frameworks. Furthermore, an expansion of the method to the multiple change-point framework is discussed through an idea similar to that of moving sum (MOSUM; Eichinger & Kirch, 2018). It would be beneficial to the reader for the authors to justify this choice; is it due to MOSUM’s low computational complexity? *(abstract via OpenAlex)*

JRSSB Machine Learning Econometrics & Economics 讨论 / 评论 · 发表于 2023-12-28

Yingnian Wu and Weng Kee Wong’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Ying Nian Wu, Weng Kee Wong

摘要

We thank the authors for the interesting and timely article. Our commentaries are as follows: From equilibrium Langevin dynamics to non-equilibrium ordinary and stochastic differential equations. We wish to add a little background explanation, with no claim of originality, to this deep and elegant paper that greatly expands the scope of diffusion/score-based models. The diffusion-based model can be viewed as a refined variational auto-encoder and a non-equilibrium Langevin sampler. While the variational auto-encoder perspective seems more fundamental because it directly targets the log-likelihood through the variational lower bound, the non-equilibrium Langevin perspective is also revealing. Specifically, for a target density p(x)⁠, the Langevin dynamics for sampling p(x) is where Δt is discretized time step, and ϵt∼N(0,I) independently over discretized t (more rigorous treatment involves taking the limit Δt→0⁠). The Langevin dynamics consists of a deterministic gradient ascent term and a stochastic noise injection term. It is an equilibrium sampler in that if xt∼p(x)⁠, then xt+Δt∼p(x)⁠. However, for Markov chain Monte Carlo sampling, we have to start from a simple initial noise distribution and iterate the Langevin dynamics for infinite time in order to converge to p(x)⁠. But the familiar Langevin dynamics tells us a simple yet profound fact if we separate the deterministic gradient ascent step and the stochastic noise injection step. Suppose x∼p(x)⁠, let x−=x+Δt2∇xlogp(x)∼p−(x) by gradient ascent. Then x−+Δtϵ∼p(x) after noise injection, where ϵ∼N(0,I)⁠. Put it another way, if x−∼p−(x)⁠, and x−+Δtϵ∼p(x) after noise injection, then x+Δt2∇xlogp(x)∼p−(x)⁠. That is, the deterministic gradient ascent step reverses the noise injection step as far as the marginal distribution is concerned. This suggests that if we start from the data distribution x0∼p0(x)⁠, and iterates the noise injection step xt+Δt∼xt+Δtϵt⁠, then we can reverse it by the gradient ascent step x~t−Δt=x~t+Δt2∇xlogpt(x~t)⁠, where pt is the density of xt⁠, and we use x~ notation for the samples along the reverse trajectory, where x~t has the same marginal distribution as xt⁠. This leads to denoising by ordinary differential equation. If we double the step size of gradient ascent, then x~t+Δt∇xlogpt(x~t) amounts to x~t−2Δt⁠, and we need to inject noise Δtϵ~t to get back to the marginal distribution at t−Δt⁠. This leads to denoising by stochastic differential equation: x~t−Δt=x~t+Δt∇xlogpt(x~t)+Δtϵ~t⁠. Both ordinary and stochastic differential equations are finite-time non-equilibrium samplers. The authors replied later, in writing, as follows: *(abstract via OpenAlex)*

JRSSB Machine Learning Spatial & Time Series Networks & Graphs 讨论 / 评论 · 发表于 2023-12-28

Hernando Ombao’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Hernando Ombao

摘要

I congratulate the authors for their fundamental contribution that demonstrates how the existing tests for change-points can be represented by a neural network. The major consequence is that with sufficient data, the performance of these neural networks should be at par with these existing tests. This formulation is very insightful and inspired me to revisit the previous work on models for non-stationary time series that use time-localized representations such as wavelets, wavelet packets (WP), and the smooth localized complex exponentials (SLEXs). Orthonormal representations are mathematically elegant and they generalize the Cramer representation for stationary time series which use the Fourier basis functions. One advantage of the WP and SLEX is that they form a library of many bases and thus provide a collection of candidate representations. The best representation is selected data-adaptively to be the one that minimizes the penalized Kullback–Leibler criterion. Due to the localized nature of the basis functions, the best model gives a specific segmentation of the time series data (in the case of SLEX, this is the segmentation of time). Thus, selecting the best model implicitly provides the estimated change-points in the tine series data. The current implementation limits the choice of the model to have a dyadic segmentation of time (in the case of SLEX) or frequency (in the case of WPs). This dyadic constraint is necessary to take advantage of the best basis algorithm of Coifman and Wickerhauser which requires the magnitude of O(T) operations (where T is the number of time points). The dyadic nature is a primary limitation of these methods because the change-points are constrained to lie on dyadic time points. This is where the current work of Li et al. can provide a major improvement. The dyadic change-points could serve as the initial values of these change-points but could be refined and made more precise by taking advantage of the neural networks. *(abstract via OpenAlex)*

SMMR Longitudinal Data High-Dimensional · 发表于 2023-12-28

Fixed and random effect selections in generalized linear mixed models

Shou-En Lu, Sinae Kim, Jerry Q Cheng, Changfa Lin, Sharad Goyal, Salma K Jabbour

摘要

Generalized linear mixed models are commonly used to describe relationships between correlated responses and covariates in medical research. In this paper, we propose a simple and easily implementable regularized estimation approach to select both fixed and random effects in generalized linear mixed model. Specifically, we propose to construct and optimize the objective functions using the confidence distributions of model parameters, as opposed to using the observed data likelihood functions, to perform effect selections. Two estimation methods are developed. The first one is to use the joint confidence distribution of model parameters to perform simultaneous fixed and random effect selections. The second method is to use the marginal confidence distributions of model parameters to perform the selections of fixed and random effects separately. With a proper choice of regularization parameters in the adaptive LASSO framework, we show the consistency and oracle properties of the proposed regularized estimators. Simulation studies have been conducted to assess the performance of the proposed estimators and demonstrate computational efficiency. Our method has also been applied to two longitudinal cancer studies to identify demographic and clinical factors associated with patient health outcomes after cancer therapies.

SiM Machine Learning Optimization & Learning Theory · 发表于 2023-12-28

Fetal growth analysis from ultrasound videos based on different biometrics using optimal segmentation and hybrid classifier

B. Devisri, M. Kavitha

摘要

Birth defects and their associated deaths, high health and financial costs of maternal care and associated morbidity are major contributors to infant mortality. If permitted by law, prenatal diagnosis allows for intrauterine care, more complicated hospital deliveries, and termination of pregnancy. During pregnancy, a set of measurements is commonly used to monitor the fetal health, including fetal head circumference, crown‐rump length, abdominal circumference, and femur length. Because of the intricate interactions between the biological tissues and the US waves mother and fetus, analyzing fetal US images from a specialized perspective is difficult. Artifacts include acoustic shadows, speckle noise, motion blur, and missing borders. The fetus moves quickly, body structures close, and the weeks of pregnancy vary greatly. In this work, we propose a fetal growth analysis through US image of head circumference biometry using optimal segmentation and hybrid classifier. First, we introduce a hybrid whale with oppositional fruit fly optimization (WOFF) algorithm for optimal segmentation of segment fetal head which improves the detection accuracy. Next, an improved U‐Net design is utilized for the hidden feature (head circumference biometry) extraction which extracts features from the segmented extraction. Then, we design a modified Boosting arithmetic optimization (MBAO) algorithm for feature optimization to selects optimal best features among multiple features for the reduction of data dimensionality issues. Furthermore, a hybrid deep learning technique called bi‐directional LSTM with convolutional neural network (B‐LSTM‐CNN) for fetal growth analysis to compute the fetus growth and health. Finally, we validate our proposed method through the open benchmark datasets are HC18 (Ultrasound image) and oxford university research archive (ORA‐data) (Ultrasound video frames). We compared the simulation results of our proposed algorithm with the existing state‐of‐art techniques in terms of various metrics.

SMMR Functional Data Longitudinal Data · 发表于 2023-12-28

Assessing intra- and inter-method agreement of functional data

Ye Yue, Jeong Hoon Jang, Amita K. Manatunga

摘要

Modern medical devices are increasingly producing complex data that could offer deeper insights into physiological mechanisms of underlying diseases. One type of complex data that arises frequently in medical imaging studies is functional data, whose sampling unit is a smooth continuous function. In this work, with the goal of establishing the scientific validity of experiments involving modern medical imaging devices, we focus on the problem of evaluating reliability and reproducibility of multiple functional data that are measured on the same subjects by different methods (i.e. different technologies or raters). Specifically, we develop a series of intraclass correlation coefficient and concordance correlation coefficient indices that can assess intra-method, inter-method, and total (intra + inter) agreement based on multivariate multilevel functional data consisting of replicated functional data measurements produced by each of the different methods. For efficient estimation, the proposed indices are expressed using variance components of a multivariate multilevel functional mixed effect model, which can be smoothly estimated by functional principal component analysis. Extensive simulation studies are performed to assess the finite-sample properties of the estimators. The proposed method is applied to evaluate the reliability and reproducibility of renogram curves produced by a high-tech radionuclide image scan used to non-invasively detect kidney obstruction.

SMMR Causal Inference · 发表于 2023-12-28

Application of marginalized zero-inflated models when mediators have excess zeroes

Andrew Sims, Hemant Tiwari, Emily B. Levitan, Dustin Long, George Howard, Todd Brown, Melissa J. Smith, Jinhong Cui, D. Leann Long

摘要

Mediation analysis has become increasingly popular over the last decade as researchers are interested in assessing mechanistic pathways for intervention. Although available methods have increased, there are still limited options for mediation analysis with zero-inflated count variables where the distribution of response has a “cluster” of data at the zero value (i.e. distribution of number of cigarettes smoked per day, where nonsmokers cluster at zero cigarettes). The currently available methods do not obtain unbiased population average effects of mediation effects. In this paper, we propose an extension of the counterfactual approach to mediation with direct and indirect effects to scenarios where the mediator is a count variable with excess zeroes by utilizing the Marginalized Zero-Inflated Poisson Model (MZIP) for the mediator model. We derive direct and indirect effects for continuous, binary, and count outcomes, as well as adapt to allow mediator-exposure interactions. Our proposed work allows straightforward calculation of direct and indirect effects for the overall population mean values of the mediator, for scenarios in which researchers are interested in generalizing direct and indirect effects to the population. We apply this novel methodology to an application observing how alcohol consumption may explain sex differences in cholesterol and assess model performance via a simulation study comparing the proposed MZIP mediator framework to existing methods for marginal mediator effects.

JRSSB Machine Learning Spatial & Time Series Networks & Graphs 讨论 / 评论 · 发表于 2023-12-27

Yudong Chen and Yining Chen’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Yudong Chen, Yining Chen

摘要

We congratulate the authors for providing this stimulating work which uses deep neural networks for detecting change-points. We shall comment on three aspects of the paper: (i) the standardization procedure, (ii) distributional assumptions of the training and testing samples, and (iii) potentially more efficient use of the training samples. First, we emphasize that classification procedures based on neural networks will not automatically be invariant to shifting or scaling. To illustrate this point, we consider scenario S1 with ρt = 0.5 and with no standardization performed on the training or testing sets. Figure 1a shows the results where an independent U ∼ U[−2,2] mean-shift is added to each series from only the testing set. Without standardization, the proposed method is no longer desirable even when we increase the size of the training set, N, to 1,600. Scaling to [0,1], as proposed in the paper, will alleviate this issue. Standardization using trimmed mean and trimmed standard deviation estimates would be a more robust option. From our experiments, other approaches such as adding a random baseline to all training samples would also work. Plot of test set misclassification error rate, computed on a test set of size 150,000, against training sample size n for detecting the existence of a change-point on data series of length n = 100 under scenario S1 described in Section 5 of the paper, with the exception that ρt = 0.5. We compare the performance of the CUSUM test and neural networks from four function classes as specified in Section 5. Here we use batch size of 32 for training with no regularization. For standardization, we use the approach suggested in the paper. For the testing set used to produce (a), an independent mean-shift of U[−2,2] is added to each series. (a) µL = 0 for the training samples, µL ∼ U[−2,2] for the testing samples; data not standardized. (b) µL = 0 for both training and testing; data not standardized. (c) µL = 0 for both training and testing; data standardized. (d) µL = 0 for both training and testing; data standardized; no reversed samples added. (e) µL = 0 for both training and testing; data standardized; reversed samples added. Nevertheless, if the distributions of the training and test samples, denoted by Dtrain and D, are indeed the same, then performing standardization might lead to worse performance, as is illustrated in Figure 1b and 1c. We suspect that it is because of certain distributions change-point can be characterized by statistics that are not invariant to shifting or scaling. These statistics are likely simpler to learn than the Cumulative Sum (CUSUM). Although, as demonstrated before, without standardization, the resulting classifier might not be transferable to even slightly different settings. Besides, the presented theory requires Dtrain = D. We anticipate similar results to hold if Dtrain is a finite mixture with one component being D. More broadly speaking, consistency should hold if the support of Dtrain contains that of D. In addition, one could use different labels for different types of change-points (e.g. change in mean/variance/etc.) in the training set, and apply the existing approach to learn a multi-class classifier. Finally, we believe that the training samples can be used in a more efficient manner with a minor twist. For X1, X2, …, Xn with label Y, its reversed sequence Xn, Xn−1, …, X1 should also have the same label. Consequently, we could double the size of N by adding all the reversed sequences into the training set. Sizeable improvement with this extra step can be seen, especially when N is small, by comparing Figure 1d with Figure 1e. *(abstract via OpenAlex)*

JRSSB Machine Learning Networks & Graphs Meta-Analysis & Data Integration 讨论 / 评论 · 发表于 2023-12-27

Pierre-Aurelien Gilliot, Christophe Andrieu, Anthony Lee, Song Liu, and Michael Whitehouse’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Pierre-Aurelien Gilliot, Christophe Andrieu, Anthony Lee, Song Liu, Michael Whitehouse

摘要

We would like to thank the authors for their interesting contribution and the clarification brought on a particular aspect of diffusion models concerned with the criterion used in the fitting phase of the procedure. Another aspect, not discussed extensively, is the choice of the neural network used to model the gradients involved in the generative phase. In particular, the implicit prior information this induces on the class of distributions considered may seem mysterious and perhaps not always controllable? Here, we report results from a toy numerical experiment, which seem to raise some questions. We consider 28×28 images, in which all but four pixels have values distributed according to a Beta(10,10), re-scaled between 0 and 0.49⁠. The four remaining pixels have values distributed according to a Beta(10,10), re-scaled between 0.51 and 1, and are organised in two distinct configurations: square or diamond (Figure 1). Squares are uniformly distributed across the whole image, while diamonds are uniformly distributed only in the bottom right quadrant. Four different datasets were created, each comprising 82,000 of such images, with varying proportions of squares and diamonds (Figure 2), and were each used to train four different U-nets approximating the score function following standard practice (Ho et al., 2020; HuggingFace, 2023). Configurations in the training dataset: a) squares and b) diamonds. The grey shading represents the bottom right quadrant. Images are binarised using a threshold set at 0.5 for easier visualisation. Frequency of diamonds. The grey bar represents the frequency of diamonds generated inside the bottom right quadrant while white bar indicates the frequency of diamonds generated outside of it. A generative model which is coherent with this dataset should be expected to generate images with squares or diamonds, respecting pixel boundaries for the diamonds, and matching the frequencies of each configuration. Remarkably, most sampled images ( 94%⁠) correctly consist of either squares or diamonds. Diamonds were not confined to the bottom right quadrant anymore (Figure 2), which could stem from the shift-equivariant property of the U-Net’s convolutional layers (Cohen & Welling, 2016). However, this may not be a desirable feature. Diffusion modelling aims at alleviating the challenges faced by score matching when dealing with disjointed, multimodal distributions such as the mixture square/diamond considered here (Song & Ermon, 2019). However, we encountered difficulties in accurately sampling according to the training set mixture weights: the sampled diamond frequencies did not reflect the training set diamond frequencies across the various datasets (Figure 2). Interestingly, these sampled frequencies differed when using another seed to initialise the U-Net weights (but were still unexpected). While the link between simple properties of the neural network chosen and the class of distributions implicitly defined can sometimes be understood (Yim et al., 2023), a general characterisation is missing from the literature. This phenomenon is further compounded by additional unknown neural network properties. We are curious about the current understanding of this matter. *(abstract via OpenAlex)*

JRSSB Bayesian Methods Machine Learning 讨论 / 评论 · 发表于 2023-12-27

Gilbert MacKenzie’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Gilbert MacKenzie

摘要

I would like first to congratulate both authors on presenting their very stimulating work in the spirit of the conference’s theme of statisticians and machine learners working in alliance. There is no doubt in my mind that collaboration will lead to better, more parsimonious, models. My comments relate, mainly, to the second discussant’s points on over-parametrization. If a feed-forward neural net (Ripley, 1994) is denoted by Net(x,θ)⁠, where x is the input layer and ϕ is the defining parameter vector, colleagues, in the University of Limerick, cast the problem in a standard non-linear regression mould, thus: where ϵi∼N(0,σ2) (McInerney & Burke, 2022a). This leads to standard Gaussian loge likelihood, ℓ(θ)⁠, for inference (here θ represents the collection of all net parameters to be estimated). Standard theory leads to θ^∼N(θ,Σ) where Σ^=Io(θ^)−1⁠, where Io(θ^) is the observed, positive definite, information matrix, whence the uncertainty in θ can be assessed. Some care is required to avoid the singularities in Σ^ caused by over-fitting, e.g. when dealing with redundant nodes. Given this probability model and its likelihood function, the model space is searched using the Bayesian information criterion. The choice of Bayesian information criterion, rather than the conventional out-of-sample performance criterion for model selection, leads to an increased probability of recovering the true (more parsimonious) model with comparable, or better, out-of-sample performance. A three-stage, backwards, model selection strategy is employed: it first reduces the number of hidden nodes (architectural pruning), then inputs (variable selection), and ends with a final adjustment (fine-tuning). The availability of the variance–covariance matrix opens up opportunities to (a) frame the output of the neural net analysis in a more familiar, regression-like, format and (b) test for the existence of known or hypothetical structures in the data, for example, as in genetic analysis. There are also other modelling opportunities. Analyses carried out to date are promising; in some cases, comparable performance is obtained with half of the parameters required by the standard neural net. This work is at an early stage and the neural nets studied so far are less complicated than those discussed today. However, as the project progresses (McInerney & Burke, 2022b), it aims to cover the neural net piste. Meanwhile, we must not overlook the unresolved problem of ‘explanation’. *(abstract via OpenAlex)*

SiM Causal Inference · 发表于 2023-12-27

Analyzing risk factors for post‐acute recovery in older adults with Alzheimer's disease and related dementia: A new semi‐parametric model for large‐scale medicare claims

Biyi Shen, Haoyu Ren, Michelle Shardell, Jason Falvey, Chixiang Chen

摘要

Nearly 300,000 older adults experience a hip fracture every year, the majority of which occur following a fall. Unfortunately, recovery after fall‐related trauma such as hip fracture is poor, where older adults diagnosed with Alzheimer's disease and related dementia (ADRD) spend a particularly long time in hospitals or rehabilitation facilities during the post‐operative recuperation period. Because older adults value functional recovery and spending time at home versus facilities as key outcomes after hospitalization, identifying factors that influence days spent at home after hospitalization is imperative. While several individual‐level factors have been identified, the characteristics of the treating hospital have recently been identified as contributors. However, few methodological rigorous approaches are available to help overcome potential sources of bias such as hospital‐level unmeasured confounders, informative hospital size, and loss to follow‐up due to death. This article develops a useful tool equipped with unsupervised learning to simultaneously handle statistical complexities that are often encountered in health services research, especially when using large administrative claims databases. The proposed estimator has a closed form, thus only requiring light computation load in a large‐scale study. We further develop its asymptotic properties with stabilized inference assisted by unsupervised clustering. Extensive simulation studies demonstrate superiority of the proposed estimator compared to existing estimators.

SiM Clinical Trials Survey & Sampling · 发表于 2023-12-26

Using leave‐one‐out cross validation (LOO) in a multilevel regression and poststratification (MRP) workflow: A cautionary tale

Swen Kuh, Lauren Kennedy, Qixuan Chen, Andrew Gelman

摘要

In recent decades, multilevel regression and poststratification (MRP) has surged in popularity for population inference. However, the validity of the estimates can depend on details of the model, and there is currently little research on validation. We explore how leave‐one‐out cross validation (LOO) can be used to compare Bayesian models for MRP. We investigate two approximate calculations of LOO: Pareto smoothed importance sampling (PSIS‐LOO) and a survey‐weighted alternative (WTD‐PSIS‐LOO). Using two simulation designs, we examine how accurately these two criteria recover the correct ordering of model goodness at predicting population and small‐area estimands. Focusing first on variable selection, we find that neither PSIS‐LOO nor WTD‐PSIS‐LOO correctly recovers the models' order for an MRP population estimand, although both criteria correctly identify the best and worst model. When considering small‐area estimation, the best model differs for different small areas, highlighting the complexity of MRP validation. When considering different priors, the models' order seems slightly better at smaller‐area levels. These findings suggest that, while not terrible, PSIS‐LOO‐based ranking techniques may not be suitable to evaluate MRP as a method. We suggest this is due to the aggregation stage of MRP, where individual‐level prediction errors average out. We validate these results by applying to the real world National Health and Nutrition Examination Survey (NHANES) data in the United States. Altogether, these results show that PSIS‐LOO‐based model validation tools need to be used with caution and might not convey the full story when validating MRP as a method.

JRSSB Testing & Uncertainty · 发表于 2023-12-26

The HulC: confidence regions from convex hulls

Arun Kumar Kuchibhotla, Sivaraman Balakrishnan, Larry Wasserman

摘要

We develop and analyse the HulC, an intuitive and general method for constructing confidence sets using the convex hull of estimates constructed from subsets of the data. Unlike classical methods which are based on estimating the (limiting) distribution of an estimator, the HulC is often simpler to use and effectively bypasses this step. In comparison to the bootstrap, the HulC requires fewer regularity conditions and succeeds in many examples where the bootstrap provably fails. Unlike sub-sampling, the HulC does not require knowledge of the rate of convergence of the estimators on which it is based. The validity of the HulC requires knowledge of the (asymptotic) median bias of the estimators. We further analyse a variant of our basic method, called the Adaptive HulC, which is fully data-driven and estimates the median bias using sub-sampling. We discuss these methods in the context of several challenging inferential problems which arise in parametric, semi-parametric, and non-parametric inference. Although our focus is on validity under weak regularity conditions, we also provide some general results on the width of the HulC confidence sets, showing that in many cases the HulC confidence sets have near-optimal width.

JRSSB Testing & Uncertainty · 发表于 2023-12-26

Simultaneous directional inference

Ruth Heller, Aldo Solari

摘要

We consider the problem of inference on the signs of n>1 parameters. We aim to provide 1−α post hoc confidence bounds on the number of positive and negative (or non-positive) parameters, with a simultaneous guarantee, for all subsets of parameters. We suggest to start by using the data to select the direction of the hypothesis test for each parameter; then, adjust the p-values of the one-sided hypotheses for the selection, and use the adjusted p-values for simultaneous inference on the selected n one-sided hypotheses. The adjustment is straightforward assuming the p-values of one-sided hypotheses have densities with monotone likelihood ratio, and are mutually independent. We show the bounds we provide are tighter (often by a great margin) than existing alternatives, and that they can be obtained by at most a polynomial time. We demonstrate their usefulness in the evaluation of treatment effects across studies or subgroups. Specifically, we provide a tight lower bound on the number of studies which are beneficial, as well as on the number of studies which are harmful (or non-beneficial), and in addition conclude on the effect direction of individual studies, while guaranteeing that the probability of at least one wrong inference is at most 0.05.

JASA High-Dimensional · 发表于 2023-12-26

Online Regularization toward Always-Valid High-Dimensional Dynamic Pricing

Chi-Hua Wang, Zhanyu Wang, Will Wei Sun, Guang Cheng

摘要

–Devising a dynamic pricing policy with always valid online statistical learning procedures is an important and as yet unresolved problem. Most existing dynamic pricing policies, which focus on the faithfulness of adopted customer choice models, exhibit a limited capability for adapting to the online uncertainty of learned statistical models during the pricing process. In this article, we propose a novel approach for designing a dynamic pricing policy based on regularized online statistical learning with theoretical guarantees. The new approach overcomes the challenge of continuous monitoring of the online Lasso procedure and possesses several appealing properties. In particular, we make the decisive observation that the always-validity of pricing decisions builds and thrives on the online regularization scheme. Our proposed online regularization scheme equips the proposed optimistic online regularized maximum likelihood pricing (OORMLP) pricing policy with three major advantages: encode market noise knowledge into pricing process optimism; empower online statistical learning with always-validity overall decision points; envelope prediction error process with time-uniform non-asymptotic oracle inequalities. This type of non-asymptotic inference results allows us to design more sample-efficient and robust dynamic pricing algorithms in practice. In theory, the proposed OORMLP algorithm exploits the sparsity structure of high-dimensional models and secures a logarithmic regret in a decision horizon. These theoretical advances are made possible by proposing an optimistic online Lasso procedure that resolves dynamic pricing problems at the process level, based on a novel use of non-asymptotic martingale concentration. In experiments, we evaluate OORMLP in different synthetic and real pricing problem settings and demonstrate that OORMLP advances the state-of-the-art methods. Supplementary materials for this article are available online. *(abstract via OpenAlex)*

JASA Nonparametric & Semiparametric · 发表于 2023-12-26

Nonparametric Statistical Inference via Metric Distribution Function in Metric Spaces

Xueqin Wang, Jin Zhu, Wenliang Pan, Junhao Zhu, Heping Zhang

摘要

The distribution function is essential in statistical inference and connected with samples to form a directed closed loop by the correspondence theorem in measure theory and the Glivenko-Cantelli and Donsker properties. This connection creates a paradigm for statistical inference. However, existing distribution functions are defined in Euclidean spaces and are no longer convenient to use in rapidly evolving data objects of complex nature. It is imperative to develop the concept of the distribution function in a more general space to meet emerging needs. Note that the linearity allows us to use hypercubes to define the distribution function in a Euclidean space. Still, without the linearity in a metric space, we must work with the metric to investigate the probability measure. We introduce a class of metric distribution functions through the metric only. We overcome this challenging step by proving the correspondence theorem and the Glivenko-Cantelli theorem for metric distribution functions in metric spaces, laying the foundation for conducting rational statistical inference for metric space-valued data. Then, we develop a homogeneity test and a mutual independence test for non-Euclidean random objects and present comprehensive empirical evidence to support the performance of our proposed methods. *(abstract via PubMed)*

JASA Bayesian Methods High-Dimensional Nonparametric & Semiparametric · 发表于 2023-12-26

In Nonparametric and High-Dimensional Models, Bayesian Ignorability is an Informative Prior

Antonio R. Linero

摘要

In problems with large amounts of missing data one must model two distinct data generating processes: the outcome process, which generates the response, and the missing data mechanism, which determines the data we observe. Under the ignorability condition of Rubin, however, likelihood-based inference for the outcome process does not depend on the missing data mechanism so that only the former needs to be estimated; partially because of this simplification, ignorability is often used as a baseline assumption. We study the implications of Bayesian ignorability in the presence of high-dimensional nuisance parameters and argue that ignorability is typically incompatible with sensible prior beliefs about the amount of confounding bias. We show that, for many problems, ignorability directly implies that the prior on the selection bias is tightly concentrated around zero. This is demonstrated on several models of practical interest, and the effect of ignorability on the posterior distribution is characterized for high-dimensional linear models with a ridge regression prior. We then show both how to build high-dimensional models that encode sensible beliefs about the confounding bias and also show that under certain narrow circumstances ignorability is less problematic. Supplementary materials for this article are available online. *(abstract via OpenAlex)*

JASA Nonparametric & Semiparametric Testing & Uncertainty Optimization & Learning Theory · 发表于 2023-12-26

Dimension Reduction for Fréchet Regression

Qi Zhang, Lingzhou Xue, Bing Li

摘要

With the rapid development of data collection techniques, complex data objects that are not in the Euclidean space are frequently encountered in new statistical applications. Fréchet regression model (Peterson & Müller 2019) provides a promising framework for regression analysis with metric space-valued responses. In this paper, we introduce a flexible sufficient dimension reduction (SDR) method for Fréchet regression to achieve two purposes: to mitigate the curse of dimensionality caused by high-dimensional predictors, and to provide a visual inspection tool for Fréchet regression. Our approach is flexible enough to turn any existing SDR method for Euclidean ( X , Y ) into one for Euclidean X and metric space-valued Y . The basic idea is to first map the metric space-valued random object Y to a real-valued random variable f ( Y ) using a class of functions, and then perform classical SDR to the transformed data. If the class of functions is sufficiently rich, then we are guaranteed to uncover the Fréchet SDR space. We showed that such a class, which we call an ensemble, can be generated by a universal kernel (cc-universal kernel). We established the consistency and asymptotic convergence rate of the proposed methods. The finite-sample performance of the proposed methods is illustrated through simulation studies for several commonly encountered metric spaces that include Wasserstein space, the space of symmetric positive definite matrices, and the sphere. We illustrated the data visualization aspect of our method by the human mortality distribution data from the United Nations Databases. *(abstract via PubMed)*

JASA Nonparametric & Semiparametric Latent Variables & Mixtures Bayesian Methods · 发表于 2023-12-26

Bayesian Spline-Based Hidden Markov Models with Applications to Actimetry Data and Sleep Analysis

Sida Chen, Bärbel Finkenstädt

摘要

B-spline-based hidden Markov models employ B-splines to specify the emission distributions, offering a more flexible modelling approach to data than conventional parametric HMMs. We introduce a Bayesian framework for inference, enabling the simultaneous estimation of all unknown model parameters including the number of states. A parsimonious knot configuration of the B-splines is identified by the use of a trans-dimensional Markov chain sampling algorithm, while model selection regarding the number of states can be performed based on the marginal likelihood within a parallel sampling framework. Using extensive simulation studies, we demonstrate the superiority of our methodology over alternative approaches as well as its robustness and scalability. We illustrate the explorative use of our methods for data on activity in animals, i.e. whitetip-sharks. The flexibility of our Bayesian approach also facilitates the incorporation of more realistic assumptions and we demonstrate this by developing a novel hierarchical conditional HMM to analyse human activity for circadian and sleep modelling. *(abstract via OpenAlex)*

SiM Genetics & Omics Bayesian Methods Diagnostics & Prediction · 发表于 2023-12-26

An integrated Bayesian framework for multi‐omics prediction and classification

Himel Mallick, Anupreet Porwal, Satabdi Saha, Piyali Basak, Vladimir Svetnik, Erina Paul

摘要

With the growing commonality of multi‐omics datasets, there is now increasing evidence that integrated omics profiles lead to more efficient discovery of clinically actionable biomarkers that enable better disease outcome prediction and patient stratification. Several methods exist to perform host phenotype prediction from cross‐sectional, single‐omics data modalities but decentralized frameworks that jointly analyze multiple time‐dependent omics data to highlight the integrative and dynamic impact of repeatedly measured biomarkers are currently limited. In this article, we propose a novel Bayesian ensemble method to consolidate prediction by combining information across several longitudinal and cross‐sectional omics data layers. Unlike existing frequentist paradigms, our approach enables uncertainty quantification in prediction as well as interval estimation for a variety of quantities of interest based on posterior summaries. We apply our method to four published multi‐omics datasets and demonstrate that it recapitulates known biology in addition to providing novel insights while also outperforming existing methods in estimation, prediction, and uncertainty quantification. Our open‐source software is publicly available at https://github.com/himelmallick/IntegratedLearner .

Biometrika Testing & Uncertainty · 发表于 2023-12-22

On selection and conditioning in multiple testing and selective inference

Jelle J Goeman, Aldo Solari

摘要

Summary: We investigate a class of methods for selective inference that condition on a selection event. Such methods follow a two-stage process. First, a data-driven collection of hypotheses is chosen from some large universe of hypotheses. Subsequently, inference takes place within this data-driven collection, conditioned on the information that was used for the selection. Examples of such methods include basic data splitting as well as modern data-carving methods and post-selection inference methods for lasso coefficients based on the polyhedral lemma. In this article, we take a holistic view of such methods, considering the selection, conditioning and final error control steps together as a single method. From this perspective, we demonstrate that multiple testing methods defined directly on the full universe of hypotheses are always at least as powerful as selective inference methods based on selection and conditioning. This result holds true even when the universe is potentially infinite and only implicitly defined, such as in the case of data splitting. We provide general theory and intuition before investigating in detail several case studies where a shift to a nonselective or unconditional perspective can yield a power gain.

JASA Longitudinal Data · 发表于 2023-12-22

Higher-Order Expansions and Inference for Panel Data Models

Jiti Gao, Bin Peng, Yayi Yan

摘要

In this article, we propose a simple inferential method for a wide class of panel data models with a focus on such cases that have both serial correlation and cross-sectional dependence.In order to establish an asymptotic theory to support the inferential method, we develop some new and useful higher-order expansions, such as Berry-Esseen bound and Edgeworth Expansion, under a set of simple and general conditions.We further demonstrate the usefulness of these theoretical results by explicitly investigating a panel data model with interactive effects which nests many traditional panel data models as special cases.Finally, we show the superiority of our approach over several natural competitors using extensive numerical studies. *(abstract via OpenAlex)*

JRSSB Machine Learning Spatial & Time Series Networks & Graphs 讨论 / 评论 · 发表于 2023-12-22

Frederic Schoenberg and Weng Kee Wong’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Frederic Schoenberg, Weng Kee Wong

摘要

We congratulate the authors for an interesting article. Below are our comments: An important task moving forward will be to investigate under what conditions such algorithms are sub-optimal. For example, certainly if there is a simple parametric form and it is known, correctly, then an estimate taking advantage of this information should be expected to outperform a simple neural network algorithm that does not use this information. The proof that the CUSUM-based statistic corresponds to a neural network estimate is instructive and very nice, but this alone does not ensure that in practice, a neural network is estimated that matches the ideal estimate, even in simple cases. When estimating a change-point in a practical setting, one gets a sense that the problem ultimately boils down typically to whether the values in some time interval are sufficiently different from values in the previous time interval. I wonder if using complex models such as neural networks obscures this somewhat. There are really several different change-point problems, even within the context of estimating a single change-point, rather than multiple change-points. One problem is to pinpoint the time of a change-point, given knowledge that one has occurred in the past. Another is to evaluate in real time whether a change-point seems to have just occurred. A third is to evaluate, retrospectively, whether a change-point seems to have occurred in the past, and perhaps also jointly to pinpoint its timing. This third task is the focus of the current paper. The second task, however, is often to be performed in real time, and having a fully automatic method for accomplishing this task is typically considerably more valuable for this second problem than the others. It seems plausible that neural network algorithms might be more desirable for this second problem of real-time evaluation of whether a change-point has recently occurred, rather than the more retrospective analytical problems of identifying an optimal change-point given knowledge that one has occurred, or assessing after the fact whether or not a change-point has occurred at some time in the past. It would be interesting to see if similar results can be obtained for point processes. For disease epidemics, for example, one often obtains point process data on incidence times and it is of significant interest to determine if there is a change in the process that could indicate a significant mutation of the disease. *(abstract via OpenAlex)*

JRSSB Machine Learning 讨论 / 评论 · 发表于 2023-12-22

Dean Bodenham and Niall Adams’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Dean Bodenham, Niall Adams

摘要

We congratulate the authors for their thought-provoking paper and innovative approach for detecting changepoints in a supervised manner. The results in Table 1 and Figure 2 show that the proposed approach has good performance in terms of the misclassification error rate metric, or in other words in determining whether or not a sequence contains a changepoint. However, since a primary concern in changepoint detection is the accuracy of the localization of the changepoint, it would be interesting to see the performance of the proposed approach for offline changepoint detection metrics such as the covering metric (Arbelaez et al., 2010), where the locations of the detected changepoints are compared with the locations of the true changepoints (van den Burg & Williams, 2020). It would also be instructive to see this performance in comparison to established offline methods such as the pruned exact linear time method (Killick et al., 2012) or the wild binary segmentation method (Fryzlewicz, 2014), rather than the online cumulative sum method. Another concern is how the proposed method would handle very long sequences, for example, of length more than one million. Besides computational challenges, one might expect most methods to flag a changepoint in such sequences, while the exact location of the changepoint(s) may be more difficult to determine. Finally, we suggest that one potential area of application for the proposed method may be in cellular biology where cells are tracked and their protein expression levels are recorded. These time series are often short in length, potentially N≤20⁠, which can be a challenging scenario for traditional, unsupervised changepoint detection methods, although for this application it may be possible to obtain labelled examples from experts for the necessary training. We look forward to future developments of the proposed approach. *(abstract via OpenAlex)*

Biometrika Networks & Graphs · 发表于 2023-12-22

Central limit theorems for local network statistics

P A Maugis

摘要

Summary: Subgraph counts, in particular the number of occurrences of small shapes such as triangles, characterize properties of random networks. As a result, they have seen wide use as network summary statistics. Subgraphs are typically counted globally, making existing approaches unable to describe vertex-specific characteristics. In contrast, rooted subgraphs focus on vertex neighbourhoods, and are fundamental descriptors of local network properties. We derive the asymptotic joint distribution of rooted subgraph counts in inhomogeneous random graphs, a model that generalizes most statistical network models. This result enables a shift in the statistical analysis of graphs, from estimating network summaries to estimating models linking local network structure and vertex-specific covariates. As an example, we consider a school friendship network and show that gender and race are significant predictors of local friendship patterns.

JASA Survival Analysis Nonparametric & Semiparametric · 发表于 2023-12-22

A Unified Nonparametric Fiducial Approach to Interval-Censored Data

Yifan Cui, Jan Hannig, Michael R. Kosorok

摘要

Censored data, where the event time is partially observed, are challenging for survival probability estimation. In this paper, we introduce a novel nonparametric fiducial approach to interval-censored data, including right-censored, current status, case II censored, and mixed case censored data. The proposed approach leveraging a simple Gibbs sampler has a useful property of being "one size fits all", i.e., the proposed approach automatically adapts to all types of non-informative censoring mechanisms. As shown in the extensive simulations, the proposed fiducial confidence intervals significantly outperform existing methods in terms of both coverage and length. In addition, the proposed fiducial point estimator has much smaller estimation errors than the nonparametric maximum likelihood estimator. Furthermore, we apply the proposed method to Austrian rubella data and a study of hemophiliacs infected with the human immunodeficiency virus. The strength of the proposed fiducial approach is not only estimation and uncertainty quantification but also its automatic adaptation to a variety of censoring mechanisms. *(abstract via OpenAlex)*

JRSSB Machine Learning Spatial & Time Series Networks & Graphs 讨论 / 评论 · 发表于 2023-12-21

Yongmiao Hong, Oliver Linton, Jiajing Sun, and Meiting Zhu’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Yongmiao Hong, Oliver Linton, Jiajing Sun, Meiting Zhu

摘要

This paper proposes a neural network-based approach for automating offline change-point detection. The authors show that cumulative sum (CUSUM) and generalized CUSUM are a special case of their neural network class. They emphasize misclassification error rates and their theoretical contribution is to establish some elegant results for these under i.i.d. unit variance Gaussian data with a possible change in mean. Their theoretical results outline the conditions on the empirical risk minimization neural network that allow it to achieve comparable performance to the classic CUSUM test for this very specific setting. The framework relies on N training data samples that are independent and identical copies where N≫n2logn, which seems like a lot of training is needed! The CUSUM test only needs the sample of size n. In many financial applications, we have a single time series {X1,…,Xn} and we are interested in when and how change points occur throughout the whole observation period. It is not clear how or why we should divide the data into training and testing samples and some guidance on this would be appreciated. In those applications, considerable care needs to be taken in how to account for time series dependence and where this is treated non-parametrically issues arise with bandwidth selection. The self-normalization method proposed by Shao (2010) does not require the choice of tuning parameter and so is also automatic, and is quite widely used for identifying change points within a given dataset (Shao & Zhang, 2010). However, this method is known to have poor power properties. Recently, Hong et al. (2024) proposed the adjusted-range-based self-normalization, instead of the usual long-run variance normalization, and this appears to work better under some long memory alternatives. The current paper avoids the normalization issue altogether by making use of training data, and perhaps where that exists their method can have advantages, but it would be interesting to know how competitive their method is on financial time series and on realistic sampling schemes with longer range dependence. The research is supported by National Natural Science Foundation of China (NSFC) grants No. 71988101 and No. 72173120. *(abstract via OpenAlex)*

SMMR Survival Analysis · 发表于 2023-12-21

Time-dependent receiver operating characteristic curve estimator for correlated right-censored time-to-event data

Kassu Mehari Beyene, Ding-Geng Chen

摘要

In clinical trials, evaluating the accuracy of risk scores (markers) derived from prognostic models for prediction of survival outcomes is of major concern. The time-dependent receiver operating characteristic curve and the corresponding area under the receiver operating characteristic curve are appealing measures to evaluate the predictive accuracy. Several estimation methods have been proposed in the context of classical right-censored data which assumes the event time of individuals are independent. In many applications, however, this may not hold true if, for example, individuals belong to clusters or experience recurrent events. Estimates may be biased if this correlated nature is not taken into account. This paper is then aimed to fill this knowledge gap to introduce a time-dependent receiver operating characteristic curve and the corresponding area under the receiver operating characteristic curve estimation method for right-censored data that take the correlated nature into account. In the proposed method, the unknown status of censored subjects is imputed using conditional survival functions given the marker and frailty of the subjects. An extensive simulation study is conducted to evaluate and demonstrate the finite sample performance of the proposed method. Finally, the proposed method is illustrated using two real-world examples of lung cancer and kidney disease.

Biometrika Other · 发表于 2023-12-21

The state of cumulative sum sequential changepoint testing 70 years after Page

Alexander Aue, Claudia Kirch

摘要

Quality control charts aim at raising an alarm as soon as sequentially obtained observations of an underlying random process no longer seem to be within stochastic fluctuations prescribed by an in-control scenario. Such random processes can often be modelled using the concept of stationarity, or even independence as in most classical works. An important out-of-control scenario is the changepoint alternative, for which the distribution of the process changes at an unknown point in time. In his seminal 1954 Biometrika paper, E. S. Page introduced the famous cumulative sum control charts for changepoint monitoring. Innovatively, decision rules based on cumulative sum procedures took the full history of the process into account, whereas previous procedures were based only on a fixed and typically small number of the most recent observations. The extreme case of using only the most recent observation, often referred to as the Shewhart chart, is more akin to serial outlier than changepoint detection. Page’s cumulative sum approach, introduced seven decades ago, is ubiquitous in modern changepoint analysis, and his original paper has led to a multitude of follow-up papers in different research communities. This review is focused on a particular subfield of this research, namely nonparametric sequential, or online, changepoint tests that are constructed to maintain a desired Type-1 error as opposed to the more traditional approach seeking to minimize the average run length of the procedures. Such tests have originated at the intersection of econometrics and statistics. We trace the development of these tests and highlight their properties, mostly using a simple location model for clarity of exposition, but we also review more complex situations such as regression and time series models.

JRSSB Machine Learning Spatial & Time Series Networks & Graphs 讨论 / 评论 · 发表于 2023-12-21

Shakeel Gavioli-Akilagun’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Shakeel Gavioli-Akilagun

摘要

We congratulate the authors for the interesting and thought provoking paper. Lemma 3.2 shows that the generalized likelihood ratio (GLR) test for stability of a linear regression can be viewed as a simple feed forward neural network. However, close inspection of the lemma reveals that the setup rules out several common change point problems. Consider the piecewise polynomial regression: For ξ ’s distributed i.i.d. N(0,1)⁠, the likelihood ratio statistic for a change at location i is where Ps:eY denotes the projection of elements indexed {s,…,e} in the vector Y=(y1,…,yn)′ onto the space of (discretised) polynomials of degree p. Since (2) is a linear combination of quadratic forms, hλGLR(y)=1{maxiRi(y)>λ} clearly cannot be represented as a neural network. The Wald test for the same problem (e.g. Kim et al., 2022) likewise cannot be represented in this way. In Gavioli-Akilagun and Fryzlewicz (2023), we introduce tests based on differences of local sums of the data as simple and computationally efficient alternatives to GLR and Wald tests. Interestingly, our difference-based tests can be represented as a neural network. Consider the statistic where li=max{l∈Z∣i−l≥0and i+(p+1)l≤n}⁠. Since D(⋅) is a linear operator, hλDIF(x)=1{maxi|Di(x)|>λ} can be represented as a neural network. Using the techniques in Gavioli-Akilagun and Fryzlewicz (2023), one can show that the localization rate of Algorithm 1 for the change point in (1) is of the order where Δj=(αj−βj)⁠, p*∈argmaxj=0,…,p{|Δj|(δ/n)j}⁠, and δ=τ∧(n−τ)⁠. This is unimprovable up to the B2 term. When analysing the behaviour of neural networks on change point problems, it may be useful to think in terms of difference-based tests. The following contributions were received in writing after the meeting. *(abstract via OpenAlex)*

JRSSB Machine Learning 讨论 / 评论 · 发表于 2023-12-21

Sam Power’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Sam Power

摘要

I thank the authors for a compelling paper, which describes methodology with the potential for very wide applicability. The work approaches the task of extending denoising diffusion models to general spaces by observing that even quite exotic state spaces can still accommodate convenient Markovian evolutions. A recent note of Montanari (2023) has instead obtained generalizations by focusing on indirect ‘observation processes’ y of the signal x, which could be taken to live on a space of one’s own choice (potentially entirely unrelated to the domain of x). Could the authors comment on the advantages which they perceive of the ‘Markov perspective’ relative to the ‘observation perspective’? *(abstract via OpenAlex)*

JRSSB Machine Learning Bayesian Methods Networks & Graphs 讨论 / 评论 · 发表于 2023-12-21

Proposer of the vote of thanks and contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Darren Wilkinson

摘要

Computational statistics and machine learning (ML) are closely related, and there are many opportunities for cross-fertilization of ideas between the two fields. Both can benefit from greater interaction, and the two papers being discussed here highlight some ways that this can happen. The main focus of this paper is offline detection of a single change-point using labelled training data. Interest is in the automatic generation of new offline detection methods using neural networks whilst providing statistical guarantees of method performance. Theory is developed for a class of multi-layer perceptrons (MLPs) that directly generalize existing cumulative sum-based methods. The theory developed in the paper applies to a MLP with ReLU activation, and this basic model is amenable to analysis. However, the theory only requires a single layer, and the examples all use MLPs of constant layer width, which is rarely seen in practice. Can the authors provide practical advice on choosing network depth and layer widths sensibly and safely? In particular, are there practical issues relating to the width condition, mrmr+1=O(nlogn)? The theoretical bounds suggest the need for a lot of training data, but empirically it seems that these may be overly conservative. Do the authors have any insight into this apparent mismatch? Neural networks often work better with scaled data, and min–max scaling is used for the examples in the paper, but is this safe in the presence of heavy-tailed noise? For the application based on activity data, a more sophisticated neural network architecture is adopted for which the theoretical results provided do not directly apply. What hope is there of extending the theory to such models, and in the absence of this, what practical advice can be given? It is mentioned in the paper that in the absence of labelled data, but in the presence of a full data generating process, a simulator can be used to train the network. This is simulation-based inference (SBI) (Cranmer et al., 2020), and it is worth establishing the connection with this literature, where the use of neural networks has become a standard practice in recent years. Denoising Markov models (DMMs) are deep generative models for simulating (conditional) samples from a data distribution. Huge training data sets are required, but these can be replaced by a data generating process in the context of SBI. Typical applications are to very high dimensional data such as images, but the methods can also be used for sampling Bayesian posterior distributions. These models are very expensive both to train and sample (even relative to other deep generative models), but are considered state-of-the-art for certain problems. The paper provides a unifying framework for a broad class of models of the denoising diffusion form with fairly arbitrary state spaces. The emphasis is on continuous time, but the connection with discrete time formulations is clearly articulated. Conditional simulation is also covered and briefly discussed. The approach is to work with continuous time Markov processes on a general state space, and to formulate the (de)noising process in terms of the generator of the Markov process. The resulting optimization targets are shown to generalize several different special cases that have appeared in the literature for particular state spaces. Details of how to generate samples are missing from the main paper, but are very important in practice. The examples described in the online supplementary material seem to use approximate first-order methods based on a regular time grid, but this is probably not optimal. A benefit of formulating the models in continuous time is the possibility of using higher-order methods with adaptive time steps. At least one of the examples used a time-rescaling—might adaptive time-stepping reduce the need for this, or is that a separate issue? It is sometimes convenient to have a deterministic generation mechanism using a probability flow differential equation (Song et al., 2021). Is such an approach covered by the general DMM framework presented here? What about Schrödinger bridge (Shi et al., 2022) approaches? Everything depends on using a ‘good’ neural network architecture for the denoising process, but can anything general be said about how to choose the architecture for a given problem? Do we understand the kinds of problems for which DMMs work well? Why are not these models more widely used for SBI? Given the magnitude of the computational machinery dedicated to the problem, the g-and-k example was not especially compelling (see, e.g. Figure 8 in the online supplementary material), despite being a fairly standard low-dimensional Bayesian inference problem. Could issues be diagnosed in the absence of ground truth, and could the model be tuned to improve performance if desired? Are there examples in the literature of DMMs being used for problems with a mixed discrete and continuous state space? These two papers illustrate different aspects of the interaction between statistics and ML. From the perspective of academic statistics, we are likely to see increasing use of modern ML methods in statistical methodology. It is likely to become difficult to draw a clear line between computational statistics and ML, but this comes with challenges, since the language and culture of the two communities remain quite distinct. Programming languages also illustrate potential issues: Python is the language typically used for ML, with tensor frameworks such as TensorFlow (used for Paper 1), JAX (used for Paper 2), and Torch, but most academic statisticians currently use R by default. The opportunities for sharing ideas between statistics and ML are great and growing. The two papers presented here are important contributions in their own right, and also serve to highlight the potential benefits of narrowing the gap between the two communities. It therefore gives me great pleasure to propose the vote of thanks. *(abstract via OpenAlex)*

SiM Other · 发表于 2023-12-21

Properties of the full random‐effect modeling approach with missing covariate data

Joakim Nyberg, E. Niclas Jonsson, Mats O. Karlsson, Jonas Häggström

摘要

During drug development, a key step is the identification of relevant covariates predicting between‐subject variations in drug response. The full random effects model (FREM) is one of the full‐covariate approaches used to identify relevant covariates in nonlinear mixed effects models. Here we explore the ability of FREM to handle missing (both missing completely at random (MCAR) and missing at random (MAR)) covariate data and compare it to the full fixed‐effects model (FFEM) approach, applied either with complete case analysis or mean imputation. A global health dataset (20 421 children) was used to develop a FREM describing the changes of height for age Z‐score (HAZ) over time. Simulated datasets (n = 1000) were generated with variable rates of missing (MCAR) covariate data (0%‐90%) and different proportions of missing (MAR) data condition on either observed covariates or predicted HAZ. The three methods were used to re‐estimate model and compared in terms of bias and precision which showed that FREM had only minor increases in bias and minor loss of precision at increasing percentages of missing (MCAR) covariate data and performed similarly in the MAR scenarios. Conversely, the FFEM approaches either collapsed at 70% of missing (MCAR) covariate data (FFEM complete case analysis) or had large bias increases and loss of precision (FFEM with mean imputation). Our results suggest that FREM is an appropriate approach to covariate modeling for datasets with missing (MCAR and MAR) covariate data, such as in global health studies.

JRSSB Machine Learning 讨论 / 评论 · 发表于 2023-12-21

Kanti Mardia’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Kanti V Mardia

摘要

I enjoyed reading this paper which has so many outstanding features; this is where Modern Statistics is moving—high-level computing with sound mathematical statistical. My comments are as follows. The authors use the wrapped normal on SO(3) using the exponential map and I have some reservations related to this choice since the most common distribution used on SO(3) is the Fisher matrix distribution which has many desirable properties (see, e.g. Mardia & Jupp, 2000) whereas there are inherent singularities in this type of wrapping which I will illustrate. Since SO(3) can be identified with S3 (after identifying antipodal points), any calculation on SO(3) can be reformulated as one on S3 and will consider the unit sphere Sq−1⁠. If g(x) is a pdf in the tangent space, x in Rq−1⁠, then the ‘wrapped’ pdf (with the exponential map on the sphere with base point at the north pole (0,…,0,1)⁠) of y=(sinθv,cosθ)⁠, where 0≤θ≤π and v is a unit vector in Rq−1⁠, can be shown to have the pdf with respect to the uniform measure on the sphere, where r1,k=θ+2πk and r2,k=2π(k+1)−θ⁠. Except for the term involving r1,0 at θ=0⁠, all the remaining terms have a singularity at θ=0 and at θ=π⁠. In particular, this wrapped distribution on the sphere cannot be unimodal. This is not a great problem if g(⋅) is highly concentrated near the origin on the tangent plane, but it can be an issue for more diffuse distributions (and particularly when used in mixtures, Section J5). I note that the construction is general: given a base point m on the Riemannian manifold and a vector x in the tangent space, the exponential map expm(x) yields a point on the manifold and we can go from g(⋅) to f(⋅)⁠. The use of the score matching methodology (SME) methodology is remarkable in our work (Mardia et al., 2016), which you quote on manifolds has been more in the classical statistical setting. On the other hand, the introduction of score matching approximation in Mardia (2018) explores totally a new direction; how to approximate ‘analytically’ a known distribution f* by another simpler distribution f from an exponential family in contrast to your data-driven method. A general solution is given in this paper: the score matching approximation (SMA) is motivated by the challenging problem of approximating the standard multivariate wrapped normal distribution by the multivariate von Mises distribution. My last point is on your examples. Your emphasis on assessing the performance of your methods is via the ground truths but perhaps in some cases such as for estimation, this may not be sufficient. I have also some queries on your example in Section J5: how did you make the choice of the number of components of the mixture M=16?; it is also not clear how much noise was added for the test example via σ2 and its effect; and are the standard errors of your estimates relevant? *(abstract via OpenAlex)*

JRSSB Machine Learning 讨论 / 评论 · 发表于 2023-12-21

Jorge Mateu's contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Jorge Mateu

摘要

Jorge Mateu; Jorge Mateu's contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’, Journal of t *(abstract via OpenAlex)*

JRSSB Machine Learning Econometrics & Economics 讨论 / 评论 · 发表于 2023-12-21

John Kent’s contribution to the Discussion of the ‘Discussion Meeting on Probabilistic and statistical aspects of machine learning’

John T Kent

摘要

I would like to thank Benton and his co-authors for a very stimulating paper. I will focus my comments on some ideas related to diffusions and directional distributions. In the context of Euclidean space, the forwards diffusion in the discussion below equation (2) is taken to be an Ornstein–Uhlenbeck process with drift b(x)=−λx⁠, and with infinitesimal variance equal to σ2⁠. The equilibrium distribution is an isotropic multivariate normal distribution, Np(0,σ2/(2λ)I)⁠. The authors take λ=1/2 and σ2=1⁠, but I wonder these parameters can be usefully viewed as tuning parameters. In particular, is it important to take σ2/(2λ) large enough for the information of interest to be distinguishable, but not so large that the information is lost in a sea of irrelevant ‘noise’? In the case of a compact Riemannian manifold, a natural choice in the forwards diffusion is to set the drift equal to 0; then the equilibrium distribution is the uniform distribution. But if the information of interest is concentrated in a small portion of the manifold, then perhaps it is helpful to consider alternative diffusions. Consider the unit sphere Sp−1 in Rp⁠. Two possible choices for the drift along a geodesic from the north pole to the south pole are b(θ)=−λsinθ and b(θ)=−λsin2θ⁠, where θ is the colatitude. The corresponding equilibrium distributions are the von Mises–Fisher distribution and the Watson distribution (a special case of the Bingham distribution), respectively (e.g. Kent, 1978). The Bingham distribution is mentioned here because there is a classic identification between S3 and SO(3) through a two-to-one mapping. Hence, any calculation on SO(3) can be recast in terms of a corresponding calculation on S3⁠. In particular, the Bingham distribution on S3 can be identified with the matrix von Mises–Fisher distribution on SO(3)⁠. Further, since the isotropic matrix Fisher distribution is similar in behaviour to the Brownian motion distribution of equation (29), and since the isotropic and anisotropic matrix Fisher distributions are easy to simulate efficiently (Kent et al., 2018), they may provide a useful alternative to equation (29) in certain settings. By the way, the wrapping procedure of Section J5 is not very satisfactory in general (except on the circle). In particular, on SO(3) it leads to a density with a singularity at the identity matrix; my colleague Kanti Mardia is giving more details. *(abstract via OpenAlex)*

JRSSB Machine Learning 讨论 / 评论 · 发表于 2023-12-21

Daniela Cialfi’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Daniela Cialfi

摘要

Daniela Cialfi; Daniela Cialfi’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’, Journa *(abstract via OpenAlex)*

JRSSB Machine Learning 讨论 / 评论 · 发表于 2023-12-21

Bo Zhang’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Bo Zhang

摘要

Bo Zhang; Bo Zhang’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’, Journal of the Roy *(abstract via OpenAlex)*

JASA High-Dimensional · 发表于 2023-12-21

Asymptotic Distribution-Free Independence Test for High-Dimension Data

Zhanrui Cai, Jing Lei, Kathryn Roeder

摘要

Test of independence is of fundamental importance in modern data analysis, with broad applications in variable selection, graphical models, and causal inference. When the data is high dimensional and the potential dependence signal is sparse, independence testing becomes very challenging without distributional or structural assumptions. In this paper, we propose a general framework for independence testing by first fitting a classifier that distinguishes the joint and product distributions, and then testing the significance of the fitted classifier. This framework allows us to borrow the strength of the most advanced classification algorithms developed from the modern machine learning community, making it applicable to high dimensional, complex data. By combining a sample split and a fixed permutation, our test statistic has a universal, fixed Gaussian null distribution that is independent of the underlying data distribution. Extensive simulations demonstrate the advantages of the newly proposed test compared with existing methods. We further apply the new test to a single cell data set to test the independence between two types of single cell sequencing measurements, whose high dimensionality and sparsity make existing methods hard to apply. *(abstract via PubMed)*

JRSSB Machine Learning 讨论 / 评论 · 发表于 2023-12-21

Andi Wang’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’

Andi Q Wang

摘要

I would like to congratulate the authors for this excellent article, which includes both careful theoretical developments at the foundations of such denoising models and exciting methodological innovations and experiments. Of course, there are still many unanswered questions in relation to these denoising Markov models. One of the most tantalizing such questions is touched on by the authors in their concluding discussion, namely the nature of the implicit biases which are present in this methodology. It appears to me that there are (at least) two potential sources of implicit bias in such denoising models. First, the use of the evidence lower bound (ELBO) training objective. Although the use of the ELBO is introduced as a concession to the fact that the true likelihood pT(x) is intractable, it has been shown in some cases that using an ELBO actually has beneficial implicit biases. For example, in the context of variational auto-encoders (VAEs), it has been show in certain settings that the use of the ELBO actually enables the VAE to disentangle independent signals (Reizinger et al., 2022). Although for score matching we can see the minimizer is sθ(x,t)=∇xlogqt(x)⁠, I wonder if this particular formulation of the objective leads to some beneficial implicit biases. Second, it is well known in the context of supervised learning that (stochastic) gradient descent often leads to solutions which generalize well (Chizat & Bach, 2020; Soudry et al., 2017)—it is thus natural to ask if a similar phenomenon is occurring in the denoising Markov models setting. Thus one wonders if, in conjunction with the ELBO, the use of gradient descent in this context is resulting in score estimates which ‘generalize’ well, providing novel samples rather than recovering the empirical distribution of the training data. And of course, the particular choice of neural network architecture and network hyperparameters will no doubt further bias the score estimate towards certain approximations. Overall, it would be fascinating to get a handle on which components or combination of components of the methodology are the main contributors towards the very impressive empirical performance. I would like to congratulate the authors again for this fantastic contribution to the literature and heartily agree that further attention is required! *(abstract via OpenAlex)*

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 页