2026 年的期刊文章 · 第 40 页

共 1756 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2026 到 2026。

按期刊、按主题看 →

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 页

JMLR Machine Learning Networks & Graphs Bayesian Methods · 发表于 2026

Nonparametric Estimation of a Factorizable Density using Diffusion Models

Hyeok Kyu Kwon, Dongha Kim, Ilsang Ohn, Minwoo Chae

摘要

In recent years, diffusion models, and more generally score-based deep generative models, have achieved remarkable success in various applications, including image and audio generation. In this paper, we view diffusion models as an implicit approach to nonparametric density estimation and study them within a statistical framework to analyze their surprising performance. A key challenge in high-dimensional statistical inference is leveraging low-dimensional structures inherent in the data to mitigate the curse of dimensionality. We assume that the underlying density exhibits a low-dimensional structure by factorizing into low-dimensional components, a property common in examples such as Bayesian networks and Markov random fields. Under suitable assumptions, we demonstrate that an implicit density estimator constructed from diffusion models adapts to the factorization structure and achieves the minimax optimal rate with respect to the total variation distance. In constructing the estimator, we design a sparse weight-sharing neural network architecture, where sparsity and weight-sharing are key features of practical architectures such as convolutional neural networks and recurrent neural networks.

JMLR Machine Learning Networks & Graphs · 发表于 2026

Nonlocal Techniques for the Analysis of Deep ReLU Neural Network Approximations

Cornelia Schneider, Mario Ullrich, Jan Vybíral

摘要

In recent work concerned with the approximation and expressive powers of deep neural networks, Daubechies, DeVore, Foucart, Hanin, and Petrova introduced a system of piecewise linear functions, which can be easily reproduced by artificial neural networks with the ReLU activation function, and showed that it forms a Riesz basis of $L_2([0, 1])$. Their work was subsequently generalized to the multivariate setting by Schneider and Vybíral. In the work at hand, we show that this system serves as a Riesz basis also for Sobolev spaces $W^s([0,1]^d)$ and Barron classes ${\mathbb B}^s([0,1]^d)$ with smoothness $0\lt s\lt 1$. We apply this fact to re-prove some recent results on the approximation of functions from these classes by deep neural networks. Our proof method avoids using local approximations and also allows us to track the implicit constants as well as to show that we can avoid the curse of dimension. Moreover, we also study how well one can approximate Sobolev and Barron functions by neural networks if only function values are known.

JMLR Other · 发表于 2026

Nonlinear function-on-function regression by RKHS

Peijun Sang, Bing Li

摘要

We propose a nonlinear function-on-function regression model where both the covariate and the response are random functions. The nonlinear regression is carried out in two steps: we first construct Hilbert spaces to accommodate the functional covariate and the functional response, and then build a second-layer Hilbert space for the covariate to capture nonlinearity. The second-layer space is assumed to be a reproducing kernel Hilbert space, which is generated by a positive definite kernel determined by the inner product of the first-layer Hilbert space for $X$--this structure is known as the nested Hilbert spaces. We develop estimation procedures to implement the proposed method, which allows the functional data to be observed at different time points for different subjects. Furthermore, we establish the convergence rate of our estimator as well as the weak convergence of the predicted response in the Hilbert space. Numerical studies including both simulations and a data application are conducted to investigate the performance of our estimator in finite sample.

JMLR Latent Variables & Mixtures Networks & Graphs · 发表于 2026

Node Regression on Latent Position Random Graphs via Local Averaging

Martin Gjorgjevski, Nicolas Keriven, Simon Barthelme, Yohann De Castro

摘要

Node regression consists in predicting the value of a graph label at a node, given observations at the other nodes. We perform a theoretical study where the graph is generated by a Latent Position Model: each node has a latent position and the probability of connection depends on the distance between latent positions. We begin by studying the simplest estimator: averaging the label at all neighboring nodes. We show that in Latent Position Models this estimator tends to a Nadaraya-Watson estimator in the latent space, with the same rate of convergence. One issue with this estimator is that it averages over all neighbors of a node, which may be too large or too small a region depending on the graph model. An alternative consists in first estimating the "true" distances between latent positions, then injecting these into a classical Nadaraya-Watson estimator. This enables averaging in regions either smaller or larger than the typical graph neighborhood. We show that this method can achieve standard nonparametric rates even when the graph neighborhood is too large or too small.

JMLR Networks & Graphs Optimization & Learning Theory · 发表于 2026,收录于 2026-11-06

Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study

Hao Liu, Zecheng Zhang, Wenjing Liao, Hayden Schaeffer

摘要

Neural scaling laws play a pivotal role in the performance of deep neural networks and have been observed in a wide range of tasks. However, a complete theoretical framework for understanding these scaling laws remains underdeveloped. In this paper, we explore the neural scaling laws for deep operator networks, which involve learning mappings between function spaces, with a focus on the Chen and Chen style architecture. These approaches, which include the popular Deep Operator Network (DeepONet), approximate the output functions using a linear combination of learnable basis functions and coefficients that de- pend on the input functions. We establish a theoretical framework to quantify the neural scaling laws by analyzing its approximation and generalization errors. We articulate the re- lationship between the approximation and generalization errors of deep operator networks and key factors such as network model size and training data size. Moreover, we address practical cases where input functions exhibit low-dimensional structures, allowing us to derive tighter error bounds. These results also hold for deep ReLU networks and other similar network structures. Our results offer a partial explanation of the neural scaling laws in operator learning and provide a theoretical foundation for their applications.

JMLR Networks & Graphs Optimization & Learning Theory Causal Inference · 发表于 2026

Neural Network Parameter-optimization of Gaussian Pre-marginalized Directed Acyclic Graphs

Mehrzad Saremi

摘要

Finding the parameters of a latent variable causal model is central to causal inference and causal identification. In this article, we show that existing graphical structures that are used in causal inference are not stable under marginalization of Gaussian Bayesian networks, and present a graphical structure that faithfully represents margins of Gaussian Bayesian networks. We present the first duality between parameter optimization of a latent variable model and training a feed-forward neural network in the parameter space of the assumed family of distributions. Based on this observation, we develop an algorithm for parameter optimization of these graphical structures using the observational distribution. Then, we provide conditions for causal effect identifiability in the Gaussian setting. We propose a meta-algorithm that checks whether a causal effect is identifiable or not. Moreover, we lay a grounding for generalizing the duality between a neural network and a causal model from the Gaussian to other distributions.

JMLR Optimization & Learning Theory · 发表于 2026

Neural Exploitation and Exploration of Contextual Bandits

Yikun Ban, Yuchen Yan, Arindam Banerjee, Jingrui He

摘要

In this paper, we study the neural exploration strategy for contextual bandits. The dilemma of exploitation and exploration widely exists in real-world applications such as recommender systems, online advertising, and clinical trials. Contextual bandits provide principled methods to solve this dilemma, including two prevalent techniques: Thompson Sampling (TS), and Upper Confidence Bound (UCB). Neural contextual bandits have been studied to adapt to the non-linear reward function, combined with TS or UCB strategies for exploration. In this paper, we introduce, EE-Net, which is a novel framework to utilize another neural network to learn the potential gain of exploitation neural network for exploration, different from UCB-based and TS-based approaches that rely on the large-deviation-based statistical confidence bound. In addition, we provide an instance-based $\widetilde{\mathcal{O}}(\sqrt{T})$ regret upper bound for EE-Net with a new proof workflow. Empirically, we show that EE-Net outperforms related linear and neural contextual bandit baselines on real-world datasets.

SiM Survival Analysis Networks & Graphs Nonparametric & Semiparametric · 发表于 2026,收录于 2026-08-17

Network Meta-Analysis of Survival Outcomes With Non-Proportional Hazards Using Flexible M-Splines

David M. Phillippo, Ayman Sadek, Hugo Pedder, Nicky J. Welton

摘要

Network meta‐analysis (NMA) is widely used in healthcare decision‐making, where estimates of the effect of multiple treatments on outcomes are required. For time‐to‐event outcomes such as survival or disease progression the most common approach is to model log hazard ratios; however, this relies on the proportional hazards assumption. Novel treatments such as immunotherapies are expected to display complex hazard functions that cannot be captured by standard parametric models, which results in non‐proportional hazards when comparing treatments from different classes. As a result, alternative models such as fractional polynomials or restricted cubic splines are often used. These allow substantial flexibility on the shape of the baseline hazard, but require time‐consuming model selection or are intractable for Bayesian analysis. We propose a flexible NMA model using M‐splines on the baseline hazard, with a novel weighted random walk prior distribution that provides shrinkage to avoid overfitting and is invariant to the choice of knots and timescale. Non‐proportional hazards are modeled either by stratifying by treatment or by introducing treatment effects on the spline coefficients, and covariates may be included on the log hazard rate and spline coefficients. Treatment and covariate effects on the spline coefficients are given random walk prior distributions to smoothly model departures from proportionality over time. The methods are implemented in the user‐friendly R package multinma, which supports analyzes with aggregate data, individual participant data, or mixtures of both. We apply the methods to a NMA of progression‐free survival with treatments for non‐small cell lung cancer.

JMLR Other · 发表于 2026

Nested Subspace Learning with Flags

Tom Szwagier, Xavier Pennec

摘要

Many machine learning methods look for low-dimensional representations of the data. The underlying subspace can be estimated by first choosing a dimension q and then optimizing a certain objective function over the space of q-dimensional subspaces (the Grassmannian). Trying different q generally yields non-nested subspaces, which raises an important issue of consistency between the data representations. In this paper, we propose a simple and easily implementable principle to enforce nestedness in subspace learning methods. It consists in lifting Grassmannian optimization criteria to flag manifolds (the space of nested subspaces of increasing dimension) via nested projectors. We apply the flag trick to several classical machine learning methods and show that it successfully addresses the nestedness issue.

JMLR Other · 发表于 2026

Near-optimal Delta-convex Estimation of Lipschitz Functions

Gábor Balázs

摘要

This paper presents a tractable algorithm for estimating an unknown Lipschitz function from noisy observations and establishes an upper bound on its convergence rate. The approach extends max-affine methods from convex shape-restricted regression to the more general Lipschitz setting. A key component is a nonlinear feature expansion that maps max-affine functions into a subclass of delta-convex functions, which act as universal approximators of Lipschitz functions while preserving their Lipschitz constants. Leveraging this property, the estimator attains the minimax convergence rate (up to logarithmic factors) with respect to the intrinsic dimension of the data under squared loss and subgaussian distributions in the random design setting. The algorithm integrates adaptive partitioning to capture intrinsic dimension, a penalty-based regularization mechanism that removes the need to know the true Lipschitz constant, and a two-stage optimization procedure combining a convex initialization with local refinement. The framework is also straightforward to adapt to convex shape-restricted regression. Experiments demonstrate competitive performance relative to other theoretically justified methods, including nearest-neighbor and kernel-based regressors.

SiM Bayesian Methods · 发表于 2026,收录于 2026-08-17

Multivariable Behavioral Change Modeling of Epidemics in the Presence of Undetected Infections

Caitlin Ward, Rob Deardon, Alexandra M. Schmidt

摘要

Epidemic models are invaluable tools to understand and implement strategies to control the spread of infectious diseases, as well as to inform public health policies and resource allocation. However, current modeling approaches have limitations that reduce their practical utility, such as the exclusion of human behavioral change in response to the epidemic or ignoring the presence of undetected infectious individuals in the population. These limitations became particularly evident during the COVID‐19 pandemic, underscoring the need for more accurate and informative models. To address these challenges, we develop a novel Bayesian epidemic modeling framework to better capture the complexities of disease spread by incorporating behavioral responses and undetected infections. In particular, our framework makes three contributions: (1) leveraging additional data on hospitalizations and deaths in modeling the disease dynamics, (2) accounting for data uncertainty arising from the large presence of asymptomatic and undetected infections, and (3) allowing the population behavioral change to be dynamically influenced by multiple data sources (cases and deaths). We thoroughly investigate the properties of the proposed model via simulation, and illustrate its utility on COVID‐19 data from Montréal and Miami.

Econometrica Other · 发表于 2026

Multidimensional Screening With Precise Seller Information

Mira Frick, Ryota Iijima, Yuhta Ishii

摘要

A multi‐product monopolist faces a buyer who is privately informed about his valuations for the goods. As is well known, optimal mechanisms are in general complicated, while simple mechanisms—such as pure bundling or separate sales—can be far from optimal and do not admit clear‐cut comparisons. We show that this changes if the monopolist has sufficiently precise information about the buyer's valuations: Now, pure bundling always outperforms separate sales; moreover, there is a sense in which pure bundling performs essentially as well as the optimal mechanism. To formalize this, we characterize how fast the corresponding revenues converge to the first‐best revenue as the monopolist's information grows precise: Pure bundling achieves the same convergence rate to the first‐best as optimal mechanisms; in contrast, the convergence rate under separate sales is suboptimal.

JMLR Spatial & Time Series Networks & Graphs Latent Variables & Mixtures · 发表于 2026

Multi-relational Network Autoregression Model with Latent Group Structures

Yimeng Ren, Xuening Zhu, Ganggang Xu, Yanyuan Ma

摘要

Multi-relational networks among entities are frequently observed in the era of big data. Quantifying the effects of multiple networks has attracted significant research interest recently. In this work, we model multiple network effects through an autoregressive framework for tensor-valued time series. To characterize the potential heterogeneity of the networks and handle the high dimensionality of the time series data simultaneously, we assume a separate group structure for entities in each network and estimate all group memberships in a data-driven fashion. Specifically, we propose a group tensor network autoregression (GTNAR) model, which assumes that within each network, entities in the same group share the same set of model parameters, and the parameters differ across networks. An iterative algorithm is developed to estimate the model parameters and the latent group memberships simultaneously. Theoretically, we show that the group-wise parameters and group memberships can be consistently estimated when the group numbers are correctly or possibly over-specified. An information criterion for estimating the group number for each network is also provided to consistently select the group numbers. Lastly, we apply the GTNAR method to a Yelp dataset to illustrate its usefulness.

Econometrica Other · 发表于 2026

Monotonicity and Robust Implementation Under Forward‐Induction Reasoning

Pierpaolo Battigalli, Emiliano Catonini

摘要

In sequential games, the set of paths consistent with rationality and forward‐induction reasoning may change nonmonotonically when adding transparent restrictions on players' beliefs. Yet, we prove that—in an incomplete‐information environment—predictions become sharper when the restrictions only concern initial beliefs about types. Thus, strong rationalizability for games with payoff uncertainty characterizes the path predictions of forward‐induction reasoning across all possible restrictions on players' hierarchies of exogenous beliefs. With this, we can solve an open problem: the implementation of social choice functions through sequential mechanisms under forward‐induction reasoning—which considerably expands the realm of implementable functions compared with simultaneous mechanisms (Müller (2016))—is indeed robust in the sense of Bergemann and Morris (2009).

SiM Latent Variables & Mixtures Bayesian Methods · 发表于 2026,收录于 2026-08-17

Modeling Heterogeneity in Health Risk Behaviors With a Dynamic Mixture Model Informed by Textual Occupational Data

Lorenzo Schiavon, Mattia Stival, Angela Andreella, Stefano Campostrini

摘要

Health risk behaviors, including smoking, poor nutrition, alcohol misuse, and physical inactivity (SNAP), are leading contributors to chronic disease burden and healthcare costs worldwide. Their prevalence is shaped not only by individual demographic characteristics but also by contextual factors such as socioeconomic and occupational environments. We develop a Bayesian topic-informed dynamic mixture model to analyze SNAP behaviors using data from the Italian health and behavioral surveillance system PASSI. Free-text occupational descriptions are integrated through structural topic modeling to define latent occupational groups that inform the mixture weights of a multivariate ordered probit model. Covariate effects are allowed to vary across occupational clusters and evolve over time. To enhance interpretability and facilitate variable selection, we incorporate non-local spike-and-slab priors on regression coefficients. We further develop a tailored online learning strategy based on sequential Monte Carlo, enabling efficient updating of inference as new surveillance data become available. The proposed approach provides a flexible and interpretable framework for monitoring how occupational contexts modulate the impact of socio-demographic factors on multiple health risk behaviors over time, supporting targeted public health interventions. More broadly, the methodology is applicable to other longitudinal health surveillance and clinical settings where complex contextual information is available in textual form.

JMLR Other · 发表于 2026

Model-free generalized fiducial inference

Jonathan P Williams

摘要

Conformal prediction (CP) was developed to provide finite-sample probabilistic prediction guarantees. While CP algorithms are a relatively general purpose approach to uncertainty quantification, with finite-sample guarantees, they lack versatility. Namely, the CP approach does not prescribe how to quantify the degree to which a data set provides evidence in support of (or against) an arbitrary event from a general class of events. This paper uses tools from imprecise probability theory to build a formal connection between CP and generalized fiducial (GF) inference. These new insights establish a more general inferential lens from which CP can be understood, and demonstrate the pragmatism of fiducial ideas. The formal connection establishes a context in which epistemically-derived GF probability matches aleatoric/frequentist probability. Beyond this fact, this work illustrates how tools from imprecise probability theory, namely lower and upper probability functions, can be applied in the context of the imprecise GF distribution to provide posterior-like, prescriptive inference that is not possible within the CP framework alone. In addition to the primary CP generalization that is contributed, fundamental connections are synthesized between this new model-free GF and three other areas of contemporary research: nonparametric predictive inference (NPI), conformal predictive systems/distributions, and inferential models (IMs).

JMLR Bayesian Methods High-Dimensional · 发表于 2026

Mixing times of data-augmentation Gibbs samplers for high-dimensional probit regression

Filippo Ascolani, Giacomo Zanella

摘要

We investigate the convergence properties of popular data-augmentation samplers for Baye\-sian probit regression. Leveraging recent results on Gibbs samplers for log-concave targets, we provide simple and explicit non-asymptotic bounds on the associated mixing times (in Kullback-Leibler divergence). The bounds depend explicitly on the design matrix and the prior precision, while they hold uniformly over the vector of responses. We specialize the results for different regimes of statistical interest, when both the number of data points $n$ and parameters $p$ are large: in particular we identify scenarios where the mixing times remain bounded as $n,p\to\infty$, and ones where they do not. The results are shown to be tight (in the worst case with respect to the responses) and provide guidance on choices of prior distributions that provably lead to fast mixing. An empirical analysis based on coupling techniques suggests that the bounds are effective in predicting practically observed behaviours.

JMLR Other · 发表于 2026

Minimax density estimation in the adversarial framework under local differential privacy

Mélisande Albert, Juliette Chevallier, Béatrice Laurent, Ousmane Sacko

摘要

We consider the problem of nonparametric density estimation under privacy constraints in an adversarial framework. To this end, we study minimax rates over Sobolev spaces under local differential privacy. We first obtain a lower bound which allows us to quantify the impact of privacy compared with the classical framework. Next, we introduce a new Coordinate block privacy mechanism that guarantees local differential privacy, which, coupled with a projection estimator, achieves the minimax optimal rates. Finally, we develop an adaptive procedure which is optimal in the minimax sense up to logarithmic terms.

JMLR Optimization & Learning Theory · 发表于 2026

Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes

Ruiqi Zhang, Jingfeng Wu, Licong Lin, Peter L. Bartlett

摘要

We study gradient descent (GD) for logistic regression on linearly separable data with stepsizes that adapt to the current risk, scaled by a constant hyperparameter \(\eta\). We show that after at most \(1/\gamma^2\) burn-in steps, GD achieves a risk upper bounded by \(\exp(-\Theta(\eta))\), where \(\gamma\) is the margin of the dataset. As \(\eta\) can be arbitrarily large, GD attains an arbitrarily small risk immediately after the burn-in steps, though the risk evolution may be non-monotonic. We further construct hard datasets with margin \(\gamma\), where any batch (or online) first-order method requires \(\Omega(1/\gamma^2)\) steps to find a linear separator. Thus, GD with large, adaptive stepsizes matches the worst-case $1/\gamma^2$ dependence when the sample size is unrestricted. Notably, the classical Perceptron, a first-order online method, also achieves a step complexity of \(1/\gamma^2\), matching GD even in constants. Finally, our GD analysis extends to a broad class of loss functions and certain two-layer networks.

Econometrica Econometrics & Economics · 发表于 2026

Mechanism Design for Personalized Policy: A Field Experiment Incentivizing Exercise

Rebecca Dizon-Ross, Ariel Zucker

摘要

Personalizing policies can theoretically increase their effectiveness. However, personalization is difficult when individual types are unobservable and the preferences of policymakers and individuals are not aligned, which could cause individuals to misreport their type. Mechanism design offers a strategy to overcome this issue: offer an “incentive‐compatible” menu of policy choices designed to induce participants to select the variant intended for their type. Using a field experiment that personalized incentives for exercise among 6,800 adults with diabetes and hypertension in urban India, we show that personalizing with an incentive‐compatible choice menu substantially improves program performance, increasing the treatment effect of incentives on exercise by 80% without increasing incentive costs relative to a one‐size‐fits‐all benchmark. Offering choice achieves similar performance to personalizing with an extensive set of observable variables, but without the same data requirements.

JMLR Machine Learning Latent Variables & Mixtures · 发表于 2026,收录于 2026-09-17

MarkDiffusion: An Open-Source Toolkit for Generative Watermarking of Latent Diffusion Models

Leyi Pan, Sheng Guan, Zheyu Fu, Luyang Si, Huan Wang, Zian Wang, Hanqian Li, Xuming Hu, Irwin King, Philip S. Yu, Aiwei Liu, Lijie Wen

摘要

We introduce MarkDiffusion, an open-source Python toolkit for generative watermarking of latent diffusion models. It comprises three key components: a unified implementation framework for streamlined watermarking algorithm integrations and user-friendly interfaces; a mechanism visualization suite that intuitively showcases added and extracted watermark patterns to aid public understanding; and a comprehensive evaluation module offering standard implementations of 24 tools across three essential aspects—detectability, robustness, and output quality—plus 8 automated evaluation pipelines. Through MarkDiffusion, we seek to assist researchers, enhance public awareness and engagement in generative watermarking, and promote consensus while advancing research and applications." (JMLR's own per-paper page was blocked; retrieved from the paper's arXiv mirror, arXiv:2509.10569.) *Publication/online date: date unknown (JMLR page inaccessible; listed in Volume 27, 2026)*

Econometrica Bayesian Methods Nonparametric & Semiparametric · 发表于 2026,收录于 2026-09-24

Locally Robust Semiparametrically Efficient Bayesian Inference

Ulrich Müller, Andriy Norets

摘要

(from the author's own working-paper PDF, hosted on Ulrich Müller's Princeton University page, since onlinelibrary.wiley.com was inaccessible; title/authors closely match the published version) "We propose a framework for making Bayesian parametric models robust to local misspecification. Suppose in a baseline parametric model, a parameter of interest has an interpretation in an encompassing semiparametric model. Bayesian and maximum likelihood estimators are generally biased under local misspecification. We propose to augment the baseline likelihood by a multiplicative factor that involves scores for the baseline model, the efficient scores for the encompassing semiparametric model, and an auxiliary parameter that has the same dimension as the parameter of interest. We show that the marginal posterior for the parameter of interest in the augmented model is asymptotically normal with mean equal to the semiparametrically efficient estimator and variance equal to the semiparametric efficiency bound. The suggested augmentation robustifies the baseline parametric model to local misspecification, while preserving the appeal of Bayesian inference. We develop an MCMC algorithm for the augmented model and illustrate the approach in applications.

JMLR Other · 发表于 2026

Locally Private Estimation with Public Features

Yuheng Ma, Hanfang Yang, Ke Jia

摘要

We initiate the study of locally differentially private (LDP) learning with public features. We define semi-feature LDP, where some features are publicly available while the remaining ones, along with the label, require protection under local differential privacy. Under semi-feature LDP, we consider three fundamental estimation problems: non-parametric density estimation, classification, and regression. Given the smoothness assumption, we show that the minimax convergence rate is significantly improved compared to classical LDP. Then, we propose HistOfTree, an estimator that fully leverages the information contained in both public and private features. Theoretically, HistOfTree reaches the minimax optimal convergence rate. Empirically, HistOfTree achieves superior performance on both synthetic and real data. We also explore scenarios where users have the flexibility to select features for protection manually. In such cases, we propose an estimator and a data-driven parameter tuning strategy, leading to analogous theoretical and empirical results.

JMLR Networks & Graphs · 发表于 2026

Limiting Over-Smoothing and Over-Squashing of Graph Message Passing by Deep Scattering Transforms

Yuanhong Jiang, Dongmian Zou, Xiaoqun Zhang, Yu Guang Wang

摘要

Graph neural networks (GNNs) have become pivotal tools for processing graph-structured data, leveraging the message passing scheme as their core mechanism. However, traditional GNNs often grapple with issues such as instability, over-smoothing, and over-squashing, which can degrade performance and create a trade-off dilemma. In this paper, we introduce a discriminatively trained, multi-layer Deep Scattering Message Passing (DSMP) neural network designed to overcome these challenges. By harnessing spectral transformation, the DSMP model aggregates neighboring nodes with global information, thereby enhancing the precision and accuracy of graph signal processing. We provide theoretical proofs demonstrating the DSMP's effectiveness in mitigating these issues under specific conditions. Additionally, we support our claims with empirical evidence and thorough frequency analysis, showcasing the DSMP's superior ability to address instability, over-smoothing, and over-squashing.

JMLR Networks & Graphs Machine Learning · 发表于 2026,收录于 2026-11-06

Likelihood-based Inference in Fully and Partially Observed Exponential Family Graphical Models with Intractable Normalizing Constants

Yujie Chen, Anindya Bhadra, Antik Chakraborty

摘要

Probabilistic graphical models that encode an underlying Markov random field are fundamental building blocks of generative modeling to learn latent representations in modern multivariate data sets with complex dependency structures. Among these, the exponential family graphical models are especially popular, given their fairly well-understood statistical properties and computational scalability to high-dimensional data based on pseudo-likelihood methods. These models have been successfully applied in many fields, such as the Ising model in statistical physics and count graphical models in genomics. Another strand of models allows some nodes to be latent, so as to allow the marginal distribution of the observable nodes to depart from exponential family to capture more complex dependence. These approaches form the basis of generative models in artificial intelligence, such as the Boltzmann machines and their restricted versions. A fundamental barrier to likelihood-based (i.e., both maximum likelihood and fully Bayesian) inference in both fully and partially observed cases is the intractability of the likelihood. The usual workaround is via adopting pseudo likelihood-based approaches, following the pioneering work of Besag(1974). The goal of this paper is to demonstrate that full likelihood-based analysis of these models is feasible in a computationally efficient manner under a logarithmically sparse setting. The chief innovation lies in utilizing the tractable independence model underlying an intractable graphical model, to estimate the normalizing constant, as well as its gradient. Extensive numerical results, supporting theory and comparisons with pseudo likelihood-based approaches demonstrate the applicability of the proposed method.

JMLR Bayesian Methods · 发表于 2026,收录于 2026-11-06

Learning with Importance-weighted Variational Inference

Kamélia Daudel, François Roueff

摘要

Several variational bounds involving importance weighting ideas generalize the Evidence Lower BOund (ELBO) for marginal likelihood optimization, such as the Importance-weighted Auto-Encoder (IWAE), Variational Rényi (VR) and VR-IWAE bounds. Yet, it remains unclear how the joint choice of bound and gradient estimator impacts the behavior of the resulting variational inference (VI) algorithms. This paper provides a unified theoretical comparison of reparameterized (REP) and doubly-reparameterized (DREP) gradient estimators tied to the IWAE, VR and VR-IWAE bounds. Through asymptotic analyses of the Signal-to-Noise Ratio as the number of Monte Carlo samples $N$ goes to infinity, we identify a bias-variance tradeoff in these gradient estimators and we formally justify the superiority of DREP over REP in importance-weighted VI. An additional asymptotic analysis for challenging regimes, where both $N$ and the Kullback-Leibler divergence between the variational and posterior densities go to infinity, indicates that importance-weighted VI gradient estimators point in a well-founded direction even when the variational approximation deteriorates. Together, these complementary results characterize the optimization trajectory in importance-weighted VI from poor initialization to final convergence. Importantly, our proof techniques establish general theoretical tools for the study of sample means ratios whose scope extends beyond VI and constitute an independent contribution to the field of Monte Carlo methods.

JMLR Machine Learning · 发表于 2026

Learning to Play Two-Player Perfect-Information Games without Knowledge

Quentin Cohen-Solal

摘要

This paper introduces a set of techniques for learning game state evaluation functions through reinforcement learning. First, we generalize tree bootstrapping, i.e. learning the values of states encountered during search rather than restricting updates to states observed during matches, to the setting of reinforcement learning with non-linear function approximation. Second, we modifies Unbounded Best-First Minimax by extending best action sequences to terminal states. Third, we replace the traditional binary game outcome $+1/-1$ with richer reinforcement signals, including quick wins, delayed losses, and scoring. Fourth, we propose a completion mechanism that exploits state resolution. Finally, we introduce a novel action-selection distribution, referred to as the ordinal distribution. Experimental results show that each of these techniques contributes to substantial improvements in playing strength. We integrate them into a unified algorithm, Athénan, and compare it against ExIt, a leading self-play reinforcement learning approach without prior knowledge. Our results demonstrate that Athénan consistently outperforms ExIt. We further evaluate Athénan on the games Hex, Othello, and Arimaa, where it surpasses state-of-the-art performance without relying on domain-specific knowledge. In addition, we consider the single-player game Morpion Solitaire, in which Athénan again reaches state-of-the-art results under the same constraint. Overall, these results show that reinforcement learning, when combined with the proposed techniques, can achieve state-of-the-art performance across a diverse range of games without the need for handcrafted heuristics or expert knowledge.

JMLR High-Dimensional Nonparametric & Semiparametric Networks & Graphs · 发表于 2026

Learning general conditional independence structures via the neighbourhood lattice

Arash A. Amini, Bryon Aragam, Qing Zhou

摘要

We study the problem of learning multivariate dependencies in nonparametric and high-dimensional settings. This includes but is not limited to graphical models. Our approach effectively combines several features that are missing from previous work on this problem: We show how the entire dependence structure can be learned nonparametrically while simultaneously evading the curse of dimensionality and relaxing common assumptions such as faithfulness. To this end, we introduce and study the neighbourhood lattice decomposition of a distribution, which is a compact, non-graphical representation of conditional independence (CI) that is valid in the absence of a faithful graphical representation. We show that the neighbourhood lattice decomposition exists in any graphical model and can be computed efficiently, nonparametrically, and consistently in high-dimensions without paying the usual curse of dimensionality. This gives a way to learn all of the independence relations implied by any graphical model, without requiring a priori knowledge of the graph or even the graph type. As a special case, our results provide a general solution to the problem of nonparametric estimation of high-dimensional CI structures over any graphical model.

JMLR Bayesian Methods Networks & Graphs · 发表于 2026

Learning Bayesian Network Classifiers to Minimize Class Variable Parameters

Shouta Sugahara, Koya Kato, James Cussens, Maomi Ueno

摘要

This study proposes and evaluates a novel Bayesian network classifier which can asymptotically estimate the true probability distribution of the class variable with the fewest class variable parameters among all structures for which the class variable has no parent. Moreover, to search for an optimal structure of the proposed classifier, we propose (1) a depth-first search based method and (2) an integer programming based method. The proposed methods are guaranteed to obtain the true probability distribution asymptotically while minimizing the number of class variable parameters. Comparative experiments using benchmark datasets demonstrate the effectiveness of the proposed method.

JMLR Other · 发表于 2026

Leakage and Interpretability in Concept-Based Models

Enrico Parisini, Tapabrata Chakraborti, Chris Harbron, Ben D. MacArthur, Christopher R.S. Banerji

摘要

Concept-based Models aim to improve interpretability by predicting high-level intermediate concepts, representing a promising approach for deployment in high-risk scenarios. However, they are known to suffer from information leakage, whereby models exploit unintended information encoded within the learned concepts. We introduce an information-theoretic framework to rigorously characterise and quantify leakage, and define two complementary measures: the concepts-task leakage (CTL) and interconcept leakage (ICL) scores. We show that these measures are strongly predictive of model behaviour under interventions and outperform existing alternatives. Using this framework, we identify the primary causes of leakage and, as a case study, analyse how it manifests in Concept Embedding Models, revealing interconcept and alignment leakage in addition to the concepts-task leakage present by design. Finally, we present a set of practical guidelines for designing concept-based models to reduce leakage and ensure interpretability.

JMLR Bayesian Methods · 发表于 2026

LazyDINO: Fast, Scalable, and Efficiently Amortized Bayesian Inversion via Structure-Exploiting and Surrogate-Driven Measure Transport

Lianghao Cao, Joshua Chen, Michael Brennan, Thomas O'Leary-Roseberry, Youssef Marzouk, Omar Ghattas

摘要

We present LazyDINO, a transport map variational inference method for fast, scalable, and efficiently amortized solutions of high-dimensional nonlinear Bayesian inverse problems with expensive parameter-to-observable (PtO) maps. Our method consists of an offline phase, in which we construct a derivative-informed neural surrogate of the PtO map using joint samples of the PtO map and its Jacobian as training data. During the online phase, when given observational data, we rapidly approximate the posterior using surrogate-driven training of a lazy map, i.e., a structure-exploiting transport map with low-dimensional nonlinearity. Our surrogate construction is optimized for amortized Bayesian inversion using lazy map variational inference. We show that (i) the derivative-based reduced basis architecture minimizes an upper bound on the expected error in surrogate posterior approximation, and (ii) the derivative-informed surrogate training minimizes the expected error due to surrogate-driven variational inference. Our numerical results demonstrate that LazyDINO is highly efficient in cost amortization for Bayesian inversion. We observe a reduction of one to two orders of magnitude in offline cost for accurate online posterior approximation, compared to amortized simulation-based inference via conditional transport and to conventional surrogate-driven transport. In particular, LazyDINO consistently outperforms Laplace approximation using fewer than 1000 offline PtO map evaluations, while competing methods struggle and sometimes fail at 16,000 evaluations.

JMLR Nonparametric & Semiparametric · 发表于 2026

Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation

Luyang Fang, Haoran Lu, Yongkai Chen, Wenxuan Zhong, Ping Ma

摘要

As machine learning models and datasets continue to grow, developing complex models has become increasingly computationally demanding. Knowledge distillation reduces deployment cost by compressing a large, well-trained teacher model into a compact student model, but it does not address settings where constructing the teacher itself is the bottleneck. Motivated by this challenge, we introduce Knowledge Cascade, a reverse knowledge distillation framework that uses information from a small, inexpensive student model to guide the development of a more complex teacher model. Although this direction is counterintuitive because the teacher typically has greater representational capacity, we show that student-to-teacher transfer can be principled when supported by statistical scaling relationships. We first develop Knowledge Cascade for nonparametric multivariate functional estimation in reproducing kernel Hilbert spaces via smoothing splines, where selecting multiple smoothing parameters is a major computational bottleneck. Knowledge Cascade transfers student-selected smoothing parameters to the full-sample regime through asymptotic scaling laws, substantially reducing computational cost for high-dimensional and large-scale datasets while retaining theoretical guarantees. Beyond smoothing splines, we illustrate the same principle through kernel density estimation and deep learning hyperparameter transfer. Simulations and real-data experiments show that Knowledge Cascade achieves substantial computational savings while maintaining strong statistical performance, and can sometimes outperform the corresponding full-sample procedure.

JMLR Optimization & Learning Theory · 发表于 2026

Keypoint-Guided Optimal Transport: Models, Algorithms, and Applications

Xiang Gu, Yucheng Yang, Wei Zeng, Jian Sun, Zongben Xu

摘要

Existing Optimal Transport (OT) methods mainly derive the optimal transport plan/matching under the criterion of transport cost/distance minimization, which may cause incorrect matching in some cases. In real applications, annotating a few matched keypoints across domains is reasonable or even effortless in annotation burden. It is valuable to investigate how to leverage the annotated keypoints to guide the correct matching in OT. In this paper, we propose a novel KeyPoint-Guided model by ReLation preservation (KPG-RL) that searches for the optimal matching (i.e., transport plan) guided by the keypoints in OT. KPG-RL exploits a mask-based constraint of the transport plan to preserve the matching of keypoint pairs in transport, and guides the matching by relation of each data point to the keypoints. The KPG-RL is developed in both balanced and unbalanced/partial transport settings in Kantorovich and Gromov-Wasserstein formulations. Moreover, we deduce the dual formulation of χ²-regularized KPG-RL model, from which we learn the transport based on deep learning techniques, scaling better to larger numbers of data. With the learned transport plan, two novel neural transport strategies, named manifold barycentric projection and manifold sampling, are developed to transport source data to the target domain data manifold. As applications, we apply the proposed approach to the heterogeneous domain adaptation, multi-omic single-cell alignment, and image-to-image translation. Experiments verified the effectiveness of our approach.

JMLR Meta-Analysis & Data Integration Nonparametric & Semiparametric · 发表于 2026

Kernel-based Distributed Learning Beyond Least Squares

Heng Lian, Xu Guo

摘要

We consider one-shot distributed learning problems in a reproducing kernel Hilbert space framework. Current results are limited to the least-squares loss and extensions beyond this meet with some significant technical challenges. We establish the optimal rate of distributed learning for some general class of convex loss functions satisfying mild assumptions, using a novel empirical process on the Bregman divergence induced by the loss, which is essential for carrying out a quadratic approximation in the infinite-dimensional space. The empirical process is bounded by relating the Bregman divergence induced by the loss to the supremum norm and the $L^2$-norm of the functions. This framework incorporates many commonly used losses, including strongly smooth loss functions as well as Lipschitz continuous losses such as the quantile loss.

JMLR Nonparametric & Semiparametric Spatial & Time Series Latent Variables & Mixtures · 发表于 2026

Kernel Mean Embedding Deviation Subspace for Unsupervised Learning with Heterogeneous Data

Luoyao Yu, Lixing Zhu, Ruoqing Zhu, Xuehu Zhu

摘要

This paper proposes a method for dimension reduction that preserves information in unsupervised learning with high-dimensional heterogeneous data, specifically targeting change point detection and clustering analysis. Our main strategy is to apply a Corrected Kernel Principal Component Analysis (CKPCA) method to construct the so-called kernel mean embedding deviation subspace. The approach efficiently identifies distributional changes in these dimension reduction subspaces for unsupervised dimension reduction. For change point detection, we demonstrate that the locations and number of change points in the dimension-reduced subspaces are identical to those in the original data. Furthermore, we extend this approach to clustering by embedding the original data into nonlinear lower-dimensional spaces, providing enhanced capabilities for clustering analysis. Additionally, we explain the necessity of using CKPCA, as the classical KPCA fails to identify the kernel mean embedding deviation subspace in these problems. Numerical studies on synthetic and real data sets suggest that the dimension reduction versions of existing methods for change point detection and clustering significantly improve the performance of current approaches in finite sample scenarios.

Econometrica Econometrics & Economics · 发表于 2026

Job Ladder and Wealth Dynamics in General Equilibrium

Leo Kaas, Etienne Lalé, Nawid Siassi

摘要

This paper develops a macroeconomic model that combines an incomplete‐markets overlapping‐generations economy with a job ladder featuring sequential wage bargaining, endogenous search effort of employed and non‐employed workers, and differences in match quality. With these ingredients, our model provides a joint microfoundation for the three main inputs in aggregate production: capital, employment, and labor efficiency. The calibrated model offers a good fit to the empirical age profiles of search activity, job‐finding rates, wages, and savings. We use the model to analyze the impact of tax and transfer policies for labor market dynamics and aggregate economic activity via capital, employment, and labor efficiency channels. Lower unemployment benefits and a less progressive tax schedule bring about welfare losses for a newborn worker which are mainly driven by higher consumption risk and costlier search effort; both policies have differential effects along the age, income, and wealth dimensions.

JMLR Other · 发表于 2026

Investigating the Histogram Loss in Regression

Ehsan Imani, Kai Luedemann, Sam Scholnick-Hughes, Esraa Elelimy, Martha White

摘要

It is becoming increasingly common in regression to train neural networks that model the entire distribution even if only the mean is required for prediction. This additional modeling often comes with performance gains, and the reasons behind the improvement are not fully known. This paper investigates a recent approach to regression, the histogram loss, which involves learning the conditional distribution of the target variable by minimizing the cross-entropy between a target distribution and a flexible histogram prediction. The resulting loss corresponds to a classification loss: a cross-entropy between the outputs and a smoothed label vector. We design theoretical and empirical analyses to determine why and when this performance gain appears and how different components of the loss contribute to it. Our results suggest that the benefits of learning distributions in this setup come from improvements in optimization rather than modeling extra information. We then demonstrate the viability of the histogram loss in common deep learning applications without the need for costly hyperparameter tuning.

JMLR Other · 发表于 2026,收录于 2026-11-06

Integral Probability Metrics on Submanifolds: Interpolation Inequalities and Optimal Inference

Arthur Stéphanovitch

摘要

We study interpolation inequalities between Hölder Integral Probability Metrics (IPMs) in the case where the measures have densities on closed submanifolds. Precisely, it is shown that if two probability measures $\mu$ and $\mu^\star$ have $\beta$-smooth densities with respect to the volume measure of some submanifolds $\mathcal{M}$ and $\mathcal{M}^\star$ respectively, then the Hölder IPMs $d_{\mathcal{H}^\gamma_1}$ of smoothness $\gamma\geq 1$ and $d_{\mathcal{H}^\eta_1}$ of smoothness $\eta>\gamma$, satisfy $d_{ \mathcal{H}_1^{\gamma}}(\mu,\mu^\star)\lesssim d_{ \mathcal{H}_1^{\eta}}(\mu,\mu^\star)^\frac{\beta+\gamma}{\beta+\eta}$, up to logarithmic factors. We provide an application of this result to high-dimensional inference. These functional inequalities turn out to be a key tool for density estimation on unknown submanifold. In particular, it allows to build the first estimator attaining optimal rates of estimation for all the distances $d_{\mathcal{H}_1^\gamma}$, $\gamma \in [1,\infty)$ simultaneously.

JMLR Bayesian Methods Optimization & Learning Theory · 发表于 2026

Information-Theoretic Safe Bayesian Optimization

Alessandro G. Bottero, Carlos E. Luis, Julia Vinogradska, Felix Berkenkamp, Jan Peters

摘要

We consider a sequential decision making problem, where we aim to optimize an unknown function via noisy evaluations that do not violate an a-priori unknown (safety) constraint. As both the objective and the constraint are unknown, a common approach is to model them using a Gaussian process and restrict evaluations to those regions that are safe with high probability. Most current methods rely on a discretization of the domain and cannot be directly extended to the continuous case. Moreover, they make regularity assumptions about the safety constraint that introduce an additional critical hyperparameter. Instead, in this paper, we propose an information-theoretic safe exploration criterion that exploits solely the GP posterior to identify the safe parameters that are most informative about the safety of other parameters. The combination of this exploration criterion with a well known Bayesian optimization acquisition function yields a novel safe Bayesian optimization selection criterion. Our approach is naturally applicable to continuous domains and does not require additional explicit hyperparameters. We theoretically analyze the method and show that the proposed acquisition function can learn about the reachable safe optimum up to arbitrary precision, while only evaluating safe parameters with high probability. Empirical evaluations demonstrate data-efficiency and scalability of our approach.

Econometrica Econometrics & Economics Survival Analysis · 发表于 2026

Information Design in Common Value Auction With Moral Hazard: Application to OCS Leasing Auctions

Anh Nguyen

摘要

This paper explores the impact of information design on the auctioneer's revenue in the U.S. offshore oil/gas lease auctions where, post‐auction, the winner decides whether to explore the auctioned tract and must pay the government a royalty on its production value. I first document that there is a positive correlation between the exploration rate and publicly observed losing bids. This suggests that the winning bidder uses the rivals' bids to infer their private information about the tract's potential. I then characterize the equilibrium bidding strategy when the auctioneer designs and commits to how to reveal information on losing bids to the winning bidder. Counterfactual exercises reveal that alternative bid disclosure policies significantly improve auctioneer revenue.

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 页