2026 年的期刊文章 · 第 39 页

共 1756 篇,按发表时间排,新的在前,每页 40 篇。这一页是 2026 到 2026。

按期刊、按主题看 →

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 页

JMLR Networks & Graphs · 发表于 2026

STDE++: Polynomial-Time Amortization for Linear Differential Operators

Zekun Shi, Zheyuan Hu, Min Lin, Kenji Kawaguchi

摘要

Optimizing neural networks with losses that contain high-dimensional and high-order differential operators is expensive to evaluate with backpropagation due to $\mathcal{O}(d^{k})$ scaling of the derivative tensor size and the $\mathcal{O}(2^{k-1}L)$ scaling in the computation graph, where $d$ is the domain dimension, $L$ is the number of ops in the forward computation graph and $k$ is the derivative order. Previous works addressed the polynomial scaling in $d$ by amortizing the computation over the optimization process via randomization. Separately, the exponential scaling in $k$ for univariate functions ($d=1$) was addressed with high-order auto-differentiation (AD). In this work, we show how to efficiently perform arbitrary contractions of the derivative tensor of arbitrary order for multivariate functions by properly constructing the input tangents to univariate high-order AD, which can be used to randomize any differential operator efficiently. When applied to Physics-Informed Neural Networks (PINNs) and compared against the original PyTorch implementation of SDGD, our method yields about $1.34\times 10^{3}$ average speedup and $31.8\times$ average memory reduction across the three inseparable 100K-dimensional PDEs in our benchmark; the best case is $1.59\times 10^{3}$ speedup and $33.8\times$ memory reduction on Allen-Cahn. We can now solve 1-million-dimensional PDEs in 8 minutes on a single NVIDIA A100 GPU. Furthermore, we proposed new methods for computing mixed partial derivatives using Taylor mode AD, which scales polynomially with the derivative order. This work opens the possibility of using high-order differential operators in large-scale problems.

Econometrica Econometrics & Economics · 发表于 2026

Rural Migrants and Urban Informality: Evidence From Brazil

Clement Imbert, Gabriel Ulyssea

摘要

This paper studies the economic effects of rural‐urban migration on Brazilian cities. Using a shift‐share IV design, we show that, over a decade, drought‐induced immigration reduces informality, has no effect on unemployment, and increases the number of formal firms and jobs. Downward formal wage adjustments play a key role, as these long‐run effects are weaker in regions with stronger wage rigidity. In the short run, when wage rigidity is strongest, we replicate the informality‐increasing effects documented in the literature. We develop and estimate a model of firm dynamics and informality that rationalizes these results. The counterfactuals reveal that, in the short run, the informal sector absorbs the expanding labor force and acts as a “stepping‐stone” to formality for firms and workers. In the long run, however, it reduces the aggregate benefits from immigration by allowing the least productive firms to survive.

JMLR Machine Learning Causal Inference · 发表于 2026,收录于 2026-09-10

Robustness Against Weak or Invalid Instruments: Exploring Nonlinear Treatment Models with Machine Learning

Zijian Guo, Mengchu Zheng, Peter Bühlmann

摘要

We discuss causal inference for observational studies with possibly invalid instrumental variables. We propose a novel methodology called two-stage curvature identification (TSCI) by exploring the nonlinear treatment model with machine learning. The first-stage machine learning enables improving the instrumental variable's strength and adjusting for different forms of violating the instrumental variable assumptions. The success of TSCI requires the instrumental variable's effect on treatment to differ from its violation form. A novel bias correction step is implemented to remove bias resulting from the potentially high complexity of machine learning. Our proposed TSCI estimator is shown to be asymptotically unbiased and Gaussian even if the machine learning algorithm does not consistently estimate the treatment model. Furthermore, we design a data-dependent method to choose the best among several candidate violation forms. We apply TSCI to study the effect of education on earnings." (JMLR's own per-paper page was blocked; retrieved from the paper's arXiv mirror, arXiv:2203.12808.) *Publication/online date: date unknown (JMLR page inaccessible; listed in Volume 27, 2026)*

JMLR Machine Learning Latent Variables & Mixtures High-Dimensional · 发表于 2026

Robust training of implicit generative models for multivariate and heavy-tailed distributions with an invariant statistical loss

José Manuel de Frutos, Manuel A. Vázquez, Pablo M. Olmos, Joaquín Míguez

摘要

Implicit generative models are often trained adversarially, which can yield unstable dynamics and mode collapse. The invariant statistical loss (ISL) offers a fully sample-based alternative by comparing empirical ranks of real and generated samples. In this work, we formally characterize ISL as a proper divergence over continuous distributions and establish key regularity properties, showing that it is continuous and differentiable, thereby enabling stable gradient-based optimization without adversarial games. We further enhance ISL along two practical axes. First, to better model heavy-tailed data, where Gaussian latent priors can limit tail expressivity, we introduce Pareto-ISL, which replaces Gaussian noise with a generalized Pareto latent distribution to improve the representation of both typical and extreme events. Second, to handle multivariate data at scale, we propose ISL-slicing: a computationally efficient procedure that projects samples onto random one-dimensional subspaces, computes rank-based losses per projection, and averages them to capture high-dimensional structure. Experiments demonstrate improved tail fidelity with Pareto-ISL and show that ISL-slicing scales effectively to high dimensions. Specifically, in high dimensional settings we show that ISL can be used either as a standalone criterion or as a strong pretraining objective for subsequent adversarial fine-tuning.

JMLR Meta-Analysis & Data Integration · 发表于 2026

Resilience Beyond Stationary Client Unavailability: Unlocking Efficient and Unbiased Federated Learning

Ming Xiang, Stratis Ioannidis, Edmund Yeh, Carlee Joe-Wong, Lili Su

摘要

Due to resource constraints or external and internal uncertainties, clients in real-world federated learning systems are often intermittently available edge devices. In highly dynamic environments, the parameter server lacks prior real-time knowledge of clients' availability, making it challenging to adapt traditional federated learning algorithms to be resilient to uncertainties in client availability. If not carefully addressed, complex client availability can introduce significant bias, potentially harming the performance of the trained model. Most prior work either fails to account for non-stationary client availability dynamics or demands significant memory and computational overhead. This paper aims to develop efficient federated learning algorithms that are provably resilient to heterogeneous and non-stationary stochastic client availability. We propose FedSWE, which admits novel algorithmic structures to (i) compensate for missed computations, (ii) stabilize and diffuse the global updates over rounds, and (iii) evenly mix the local updates through implicit gossiping, despite being agnostic to non-stationary dynamics. Compared with the standard FedAvg, FedSWE introduces light additional memory and computation overhead. We show that FedSWE converges to a stationary point of non-convex objectives while achieving the desired linear speedup property in certain special cases. We corroborate our analysis with numerical experiments over diversified client unavailability dynamics on real-world data sets.

JMLR Networks & Graphs Machine Learning · 发表于 2026,收录于 2026-11-06

Residual connections provably mitigate oversmoothing in graph neural networks

Ziang Chen, Zhengjiang Lin, Shi Chen, Yury Polyanskiy, Philippe Rigollet

摘要

Graph neural networks (GNNs) have achieved remarkable empirical success in processing and representing graph-structured data across various domains. However, a significant challenge known as "oversmoothing" persists, where vertex features become nearly indistinguishable in deep GNNs, severely restricting their expressive power and practical utility. In this work, we analyze the asymptotic oversmoothing behavior of deep GNNs with and without residual connections by deriving explicit convergence rates for a normalized vertex similarity measure. Our analytical framework is grounded in the multiplicative ergodic theorem. Furthermore, we demonstrate that adding residual connections effectively mitigates or prevents oversmoothing across several broad families of parameter distributions. The theoretical findings are strongly supported by numerical experiments.

JMLR Optimization & Learning Theory Machine Learning Networks & Graphs · 发表于 2026

Reparameterized Complex-valued Neurons Can Efficiently Learn More than Real-valued Neurons via Gradient Descent

Jin-Hui Wu, Shao-Qun Zhang, Yuan Jiang, Zhi-Hua Zhou

摘要

Complex-valued neural networks potentially possess better representations and performance than real-valued counterparts when dealing with some complicated tasks such as acoustic analysis, radar image classification, etc. Despite empirical successes, it remains unknown theoretically when and to what extent complex-valued neural networks outperform real-valued ones. We take one step in this direction by comparing the learnability of real-valued neurons and complex-valued neurons via gradient descent. We theoretically show that a complex-valued neuron can learn functions expressed by any one real-valued neuron and any one complex-valued neuron with convergence rates $O(t^{-3})$ and $O(t^{-1})$ where $t$ is the iteration index of gradient descent, respectively, whereas a two-layer real-valued neural network with finite width cannot learn a single non-degenerate complex-valued neuron. We prove that a complex-valued neuron learns a real-valued neuron with rate $\Omega (t^{-3})$, exponentially slower than the linear convergence rate of learning one real-valued neuron using a real-valued neuron. We then reparameterize the phase parameter of the complex-valued neuron and prove that a reparameterized complex-valued neuron can efficiently learn a real-valued neuron with a linear convergence rate. We further verify and extend these results via simulation experiments in more general settings.

JMLR Testing & Uncertainty · 发表于 2026,收录于 2026-11-06

Reluctant Interaction Inference after Additive Modeling

Yiling Huang, Snigdha Panigrahi, Guo Yu, Jacob Bien

摘要

Additive models enjoy the flexibility of nonlinear models while still being readily understandable to humans. By contrast, other nonlinear models, which involve interactions between features, are not only harder to fit but also substantially more complicated to explain. Guided by the principle of parsimony, a data analyst therefore may naturally be reluctant to move beyond an additive model unless it is truly warranted. To put this principle of interaction reluctance into practice, we formulate the problem as a hypothesis test with a fitted sparse additive model (SPAM) serving as the null. Because our hypotheses on interaction effects are formed after fitting a SPAM to the data, we adopt a selective inference approach to construct p-values that properly account for this data adaptivity. Our approach makes use of external randomization to obtain the distribution of test statistics conditional on the SPAM fit, allowing us to derive valid p-values, corrected for the over-optimism introduced by the data-adaptive process prior to the test. Through experiments on simulated and real data, we illustrate that—even with small amounts of external randomization—this rigorous modeling approach enjoys considerable advantages over naive methods and data splitting.

JMLR Optimization & Learning Theory · 发表于 2026,收录于 2026-11-06

Regret of exploratory policy improvement and q-learning

Wenpin Tang, Xun Yu Zhou

摘要

We study the convergence of q-learning and related algorithms introduced by Jia and Zhou (J. Mach. Learn. Res., 24 (2023), 161) for controlled diffusion processes. For exploratory policy improvement, we establish exponential convergence under growth and regularity assumptions on the model parameters. For q-learning, we derive quantitative error and regret bounds under additional assumptions on the function approximation and the associated stochastic approximation dynamics.

JMLR Optimization & Learning Theory · 发表于 2026

Refined Risk Bounds for Unbounded Losses via Transductive Priors

Jian Qian, Alexander Rakhlin, Nikita Zhivotovskiy

摘要

We revisit the sequential variants of linear regression with the squared loss, classification problems with hinge loss, and logistic regression, all characterized by unbounded losses in the setup where no assumptions are made on the magnitude of design vectors and the norm of the optimal vector of parameters. The key distinction from existing results lies in our assumption that the set of design vectors is known in advance (though their order is not), a setup sometimes referred to as transductive online learning. While this assumption might seem similar to fixed design regression or denoising, we demonstrate that the sequential nature of our algorithms allows us to convert our bounds into statistical ones with random design without making any additional assumptions about the distribution of the design vectors-an impossibility for standard denoising results. Our key tools are based on the exponential weights algorithm with carefully chosen transductive (design-dependent) priors, which exploit the full horizon of the design vectors, as well as additional aggregation tools that address the possibly unbounded norm of the vector of the optimal solution. Our classification regret bounds have a feature that is only attributed to bounded losses in the literature: they depend solely on the dimension of the parameter space and on the number of rounds, independent of the design vectors or the norm of the optimal solution. For linear regression with squared loss, we further extend our analysis to the sparse case, providing sparsity regret bounds that depend additionally only on the magnitude of the response variables. We argue that these improved bounds are specific to the transductive setting and unattainable in the worst-case sequential setup. Our algorithms, in several cases, have polynomial-time approximations and reduce to sampling with respect to log-concave measures instead of aggregating over hard-to-construct epsilon-covers of classes.

JRSSB Genetics & Omics · 发表于 2026,收录于 2026-08-17

Quantiles on global non-positive curvature spaces

Ha-Young Shin, Hee-Seok Oh

摘要

(sourced from arXiv preprint, arxiv.org/abs/2312.10870 — OUP abstract page could not be loaded): "This paper develops a notion of geometric quantiles on Hadamard spaces, also known as global non-positive curvature spaces. After providing some definitions and basic properties, including scaled isometry equivariance and a necessary condition on the gradient of the quantile loss function at quantiles on Hadamard manifolds, we investigate asymptotic properties of sample quantiles on Hadamard manifolds, such as strong consistency and joint asymptotic normality. We provide a detailed description of how to compute quantiles using a gradient descent algorithm in hyperbolic space and, in particular, an explicit formula for the gradient of the quantile loss function. To demonstrate the usefulness of geometric quantiles, we define some summary measures of distributions on Hadamard manifolds with quantiles, detail outlier detection using a transformation-retransformation procedure, and implement these methodologies on both simulated and real single-cell RNA sequencing data.

JMLR Other · 发表于 2026,收录于 2026-11-06

Provable Hierarchical Martingale-based Testing for Visual Anomaly Detection

Jie Wu, Xinsong Ma, Xin Zou, Xiuwen Gong, Kaizhu Huang, Weiwei Liu

摘要

Existing visual anomaly detection approaches primarily focus on designing robust score functions but often neglect investigating the decision-making process, which plays a key role in reliable and safety-critical applications. Currently, the decision-making process suffers from two key challenges: distribution shift and theoretical guarantee. To address these issues, this paper proposes a novel approach called Hierarchical Martingale-based Testing framework (HiMT). Specifically, our approach leverages martingale theory to mitigate distribution shift issue and employs a hierarchical multiple testing procedure to rigorously control decision errors at both image level and pixel level. Extensive experimental results verify the superior decision-making capabilities of our proposed HiMT framework at both image level and pixel level.

Econometrica Other · 发表于 2026,收录于 2026-10-20

Promoting Women to Managerial Roles in the Bangladeshi Garment Sector

Rocco Macchiavello, Andreas Menzel, Atonu Rabbani, Christopher Woodruff

摘要

Women remain disadvantaged in promotion to managerial positions. We conduct a field experiment with 24 large garment factories in Bangladesh to test for inefficient representation of women among line supervisors. We identify the marginal female and male candidates for supervisory positions and randomly assign them to manage production lines. We document four findings: (1) In contrast to widespread negative beliefs about women's ability as supervisors at baseline, female candidates selected by the factories had similar skills to males; (2) during the trial, females performed worse than males, which we show is related to negative bias against them; (3) after the trial, however, many female candidates were retained as supervisors and, conditional on that, performed similar to males; and (4) after the end of our intervention, factories permanently increased the share of women among newly appointed supervisors. A conceptual framework of experimentation over discrimination rationalizes all these facts and cautions against the standard logic to test for discrimination: when there is uncertainty about the performance of the discriminated group, equal—or even worse—performance of the marginal candidates of that group is no longer sufficient to rule out inefficient discrimination.

JMLR Other · 发表于 2026,收录于 2026-11-06

Probabilistic Truly Unordered Rule Sets

Lincen Yang, Matthijs van Leeuwen

摘要

Rule set learning has long been studied because individual rules can be directly read and understood by domain experts, making them particularly valuable in high-stakes decision-making settings. Existing methods have several shortcomings though. First, most existing methods impose an order among rules, either explicitly or implicitly, which makes the comprehension of any rule's condition depend on the conditions of preceding rules. Second, due to the difficulty of handling conflicts caused by overlaps (i.e., instances covered by multiple rules), existing methods often do not consider probabilistic rules. Third, learning classification rules for a multi-class target is understudied, as most existing methods focus on either a native binary classification task, or a multi-class task decomposed into binary tasks by adopting the one-versus-rest learning strategy. To address these shortcomings, we propose TURS, for Truly Unordered Rule Sets. To resolve conflicts caused by overlapping rules, we formulate a probabilistic model class based on rule sets that encourages overlap consistency—rules that overlap should give similar probabilistic outputs. We next formalize the problem of learning a TURS model based on the minimum description length (MDL) principle and develop a carefully designed heuristic algorithm. We benchmark against a wide range of rule-based methods and demonstrate that our method learns rule sets that have lower model complexity and highly competitive predictive performance.

JMLR Nonparametric & Semiparametric Survival Analysis Spatial & Time Series · 发表于 2026

Probabilistic Rainfall Downscaling: Joint Generalized Neural Models with Censored Spatial Gaussian Copula

David Huk, Rilwan A. Adewoyin, Ritabrata Dutta

摘要

A novel approach for generating conditional probabilistic rainfall downscaling at finer scales from deterministic weather variables at coarser scales with temporal and spatial dependence is introduced. A two-step procedure is employed. Firstly, marginal location-specific distributions are jointly modelled conditional on the deterministic coarse weather variables. Secondly, a spatial dependency structure is learned to ensure spatial coherence among these distributions. To learn marginal distributions over rainfall values, we introduce joint generalised neural models that expand generalised linear models with a deep neural network architecture to jointly fit parameters of the distributions. The spatial dependency structure is modelled using a censored latent Gaussian copula leveraging the underlying spatial structure. We construct a distance matrix between locations, transformed into a correlation matrix by a Gaussian Process Kernel depending on a small set of parameters. To estimate these parameters, we propose a general framework for the estimation of latent Gaussian copulas employing scoring rules as a measure of divergence between distributions. Uniting our two contributions, namely the joint generalised neural model and the censored latent Gaussian copulas into a single model, our probabilistic approach provides downscaled rainfall. We demonstrate its efficacy using a large UK data set, outperforming existing methods.

JMLR Other · 发表于 2026,收录于 2026-09-17

Prob-GParareal: A Probabilistic Numerical Parallel-in-Time Solver for Differential Equations

Guglielmo Gattiglio, Lyudmila Grigoryeva, Massimiliano Tamborrino

摘要

We introduce Prob-GParareal, a probabilistic extension of the GParareal algorithm designed to provide uncertainty quantification for the Parallel-in-Time (PinT) solution of (ordinary and partial) differential equations (ODEs, PDEs). The method employs Gaussian processes (GPs) to model the Parareal correction function, in line with GParareal, further enabling the propagation of numerical uncertainty across time and yielding probabilistic forecasts of the system's evolution. Furthermore, Prob-GParareal accommodates probabilistic initial conditions and maintains compatibility with classical numerical solvers, ensuring its straightforward integration into existing Parareal frameworks. Here, we first conduct a theoretical analysis of the computational complexity and derive error bounds of Prob-GParareal. Then, we numerically demonstrate the accuracy and robustness of the proposed algorithm on five benchmark ODE systems, including chaotic, stiff, and bifurcation problems. To showcase the flexibility and potential scalability of the proposed algorithm, we also consider ProbnnGParareal, a variant obtained by replacing the GPs in Parareal with the nearest-neighbors GPs, illustrating its increased performance on an additional PDE example. This work bridges a critical gap in the development of probabilistic counterparts to established PinT methods." (JMLR's own per-paper page was blocked; retrieved from the paper's arXiv mirror, arXiv:2509.03945.) *Publication/online date: date unknown (JMLR page inaccessible; listed in Volume 27, 2026)*

JMLR Other · 发表于 2026

Pointwise Confidence Estimation in the Non-linear $\ell^2$-regularized Least Squares

Ilja Kuzborskij, Yasin Abbasi Yadkori

摘要

We consider a high-probability non-asymptotic confidence estimation in the $\ell^2$-regularized non-linear least-squares setting with fixed design. In particular, we study confidence estimation for local minimizers of the regularized training loss. We show a pointwise confidence bound, meaning that it holds for the prediction on any given fixed test input $x$. Importantly, the proposed confidence bound scales with similarity of the test input to the training data in the implicit feature space of the predictor (for instance, becoming very large when the test input lies far outside of the training data). This desirable last feature is captured by the weighted norm involving the inverse-Hessian matrix of the objective function, which is a generalized version of its counterpart in the linear setting, $x^{\top} \text{Cov}^{-1} x$. Our generalized result can be regarded as a non-asymptotic counterpart of the classical confidence interval based on asymptotic normality of the MLE estimator. We propose an efficient method for computing the weighted norm, which only mildly exceeds the cost of a gradient computation of the loss function. Finally, we complement our analysis with empirical evidence showing that the proposed confidence bound provides better coverage/width trade-off compared to a confidence estimation by bootstrapping, which is a gold-standard method in many applications involving non-linear predictors such as neural networks.

Econometrica Econometrics & Economics · 发表于 2026,收录于 2026-10-20

Physician Behavior in the Presence of a Secondary Market: The Case of Prescription Opioids

Molly Schnell

摘要

This paper examines how drug diversion influences the prescribing practices of physicians and the equilibrium health impacts of prescription medications. Focusing on the case of prescription opioids, a commonly prescribed and frequently diverted medication at the heart of the worst drug crisis in U.S. history, I design and estimate a model of physician behavior in the presence of a secondary market with patient search. To access prescription opioids for medical purposes or misuse, patients search over physicians on the legal primary market or turn to an illegal secondary market. Physicians, who care both about their impact on population health and their revenue from office visits, take into account the possibility that patients might resell their prescriptions on the secondary market when prescribing. The model demonstrates that the potential for diversion will tend to make strict physicians more hesitant in their prescribing while leading lenient prescribers to loosen their prescription thresholds, thereby exacerbating prescribing differences between more and less lenient physicians. Estimates reveal that the presence of a secondary market induces most physicians to be more careful in their prescribing, which brings prescriptions closer to their optimal level, but also causes harm through the reallocation of prescriptions for abuse.

JMLR Other · 发表于 2026,收录于 2026-11-06

PhiBE: A PDE-Based Bellman Equation for Continuous-Time Policy Evaluation

Yuhua Zhu

摘要

In this paper, we study policy evaluation in continuous-time reinforcement learning (RL), where the state follows an unknown stochastic differential equation (SDE) but only discrete-time data are available. We first highlight that the discrete-time Bellman equation (BE) is not always a reliable approximation to the true value function because it ignores the underlying continuous-time structure. We then introduce a new Bellman equation, PhiBE, which integrates the discrete-time information into a continuous-time PDE formulation. By leveraging the smooth structure of the underlying dynamics, PhiBE provides a provably more accurate approximation to the true value function, especially in scenarios where the underlying dynamics change slowly or the reward oscillates. Moreover, we extend PhiBE to higher orders, providing increasingly accurate approximations. We further develop a model-free algorithm for PhiBE under linear function approximation and establish its convergence under model misspecification, together with finite-sample guarantees. Directly applying classical discrete-time LSTD bounds to the continuous-time RL setting yields a model-misspecification bound with an $O({\Delta t}^{-1/2})$ stability factor and a sample-complexity bound of order $O({\Delta t}^{-2})$. By exploiting the smooth structure of the underlying dynamics, our analysis instead gives a model-misspecification bound with a stability factor independent of ${\Delta t}$ and a sample-complexity bound of order $O({\Delta t}^{-1})$. Moreover, we identify a fundamental trade-off between discretization error and sample error that is intrinsic to continuous-time policy evaluation: finer time discretization reduces bias but amplifies variance, so excessively frequent sampling does not necessarily improve performance. This is an insight that does not arise in classical discrete-time RL analyses. Numerical experiments are provided to validate the theoretical guarantees we propose.

JMLR Other · 发表于 2026

Persistence Diagrams Estimation of Multivariate Piecewise Hölder-continuous Signals

Hugo Henneuse

摘要

To our knowledge, the analysis of convergence rates for persistence diagrams estimation from noisy signals has predominantly relied on lifting signal estimation results through sup-norm (or other functional norm) stability theorems. We believe that moving forward from this approach can lead to considerable gains. We illustrate it in the setting of nonparametric regression. From a minimax perspective, we examine the inference of persistence diagrams (for the sublevel sets filtration). We show that for piecewise Hölder-continuous functions, with control over the reach of the set of discontinuities, taking the persistence diagram coming from a simple histogram estimator of the signal permits achieving the minimax rates known for Hölder-continuous functions. The key novelty lies in our use of algebraic stability instead of sup-norm stability, directly targeting the bottleneck distance through the underlying interleaving. This allows us to incorporate deformation retractions of sublevel sets to accommodate boundary discontinuities that cannot be handled by sup-norm based stability analyses.

JMLR Bayesian Methods · 发表于 2026

Particle Filter for Bayesian Inference on Privatized Data

Yu-Wei Chen, Pranav Sanghi, Jordan Awan

摘要

Differential privacy is a probabilistic framework that protects privacy while preserving data utility. To protect the privacy of the individuals in the data set, differential privacy requires adding a precise amount of noise to a statistic of interest; however, this noise addition alters the resulting sampling distribution, making statistical inference challenging. One of the main differential privacy goals in Bayesian analysis is to make statistical inference based on the private posterior distribution. While existing methods have strengths in specific conditions, they can be limited by poor mixing, strict assumptions, or low acceptance rates. We propose a novel particle filtering algorithm, which features (i) consistent estimates, (ii) Monte Carlo error estimates and asymptotic confidence intervals, (iii) computational efficiency, and (iv) accommodation to a wide variety of priors, models, and privacy mechanisms with minimal assumptions. We empirically evaluate our algorithm through a variety of simulation settings as well as an application to a 2021 Canadian census data set, demonstrating the efficacy and adaptability of the proposed sampler.

JMLR Other · 发表于 2026

Pairwise Comparisons without Stochastic Transitivity: Model, Theory and Applications

Sze Ming Lee, Yunxiao Chen

摘要

Most statistical models for pairwise comparisons, including the Bradley-Terry (BT) and Thurstone models and many extensions, make a relatively strong assumption of stochastic transitivity. This assumption imposes the existence of an unobserved global ranking among all the players/teams/items and monotone constraints on the comparison probabilities implied by the global ranking. However, the stochastic transitivity assumption does not hold in many real-world scenarios of pairwise comparisons, especially games involving multiple skills or strategies. As a result, models relying on this assumption can have suboptimal predictive performance. In this paper, we propose a general family of statistical models for pairwise comparison data without a stochastic transitivity assumption, substantially extending the BT and Thurstone models. In this model, the pairwise probabilities are determined by a (approximately) low-dimensional skew-symmetric matrix. Likelihood-based estimation methods and computational algorithms are developed, which allow for sparse data with only a small proportion of observed pairs. Theoretical analysis shows that the proposed estimator achieves minimax-rate optimality, which adapts effectively to the sparsity level of the data. The spectral theory for skew-symmetric matrices plays a crucial role in the implementation and theoretical analysis. The proposed method’s superiority against the BT model, along with its broad applicability across diverse scenarios, is further supported by simulations and real data analysis.

Econometrica Other · 发表于 2026

Outside Options, Reputations, and the Partial Success of the Coase Conjecture

Jack Fanning

摘要

A buyer and seller bargain in continuous time over a good. Bargainers can be rational or committed to some fixed price. A rational buyer has a private value and outside option. If the set of buyer values and commitment types is rich and the probability of commitment vanishes, outcomes are partially consistent with the Coase conjecture: the seller chooses a price below the maximum of the lowest outside option and half the lowest value; the buyer immediately accepts or takes his outside option.

JMLR Optimization & Learning Theory · 发表于 2026,收录于 2026-09-17

OptunaHub: A Platform for Black-Box Optimization

Yoshihiko Ozaki, Shuhei Watanabe, Toshihiko Yanase

摘要

Black-box optimization (BBO) underpins advances in domains such as AutoML and Materials Informatics, yet implementations of algorithms and benchmarks remain fragmented across research communities. We introduce OptunaHub (https://hub.optuna.org/), a community-oriented, decentralized platform for distributing BBO components under a unified Optuna-compatible interface. OptunaHub enables independent publication, discovery, and reuse of optimization algorithms and benchmark problems through a lightweight Python module, a contributor-driven registry, and a searchable web interface. The source code is publicly available in the optunahub, optunahub-registry, and optunahub-web repositories under the Optuna organization on GitHub (https://github.com/optuna/)." (JMLR's own per-paper page was blocked; retrieved from the paper's arXiv mirror, arXiv:2510.02798.) *Publication/online date: date unknown (JMLR page inaccessible; listed in Volume 27, 2026)*

JMLR Optimization & Learning Theory · 发表于 2026

Optimizing Attention with Mirror Descent: Generalized Max-Margin Token Selection

Addison Kristanto Julistiono, Davoud Ataee Tarzanagh, Navid Azizan

摘要

Attention mechanisms have revolutionized several domains of artificial intelligence, such as natural language processing and computer vision, by enabling models to selectively focus on relevant parts of the input data. While recent work has characterized the optimization dynamics of gradient descent (GD) in attention-based models and the structural properties of its preferred solutions, less is known about more general optimization algorithms such as mirror descent (MD). In this paper, we investigate the convergence properties and implicit biases of a family of MD algorithms tailored for softmax attention mechanisms, with the potential function chosen as the $p$-th power of the $\ell_p$-norm. Specifically, we show that these algorithms converge in direction to a generalized hard-margin SVM with an $\ell_p$-norm objective when applied to a classification problem using a softmax attention model. Notably, our theoretical results reveal that the convergence rate is comparable to that of traditional GD in simpler models, despite the highly nonlinear and nonconvex nature of the present problem. Additionally, we delve into the joint optimization dynamics of the key-query matrix and the decoder, establishing conditions under which this complex joint optimization converges to their respective hard-margin SVM solutions. Lastly, our numerical experiments on real data demonstrate that MD algorithms improve generalization over standard GD and excel in optimal token selection.

JMLR Optimization & Learning Theory Networks & Graphs Machine Learning · 发表于 2026

Optimization and Generalization of Gradient Descent for Shallow ReLU Networks with Minimal Width

Yunwen Lei, Puyu Wang, Yiming Ying, Ding-Xuan Zhou

摘要

Understanding the generalization and optimization of neural networks is a longstanding problem in modern learning theory. The prior analysis often leads to risk bounds of order $1/\sqrt{n}$ for ReLU networks, where $n$ is the sample size. In this paper, we present a general optimization and generalization analysis for gradient descent applied to shallow ReLU networks. We develop convergence rates of the order $1/T$ for gradient descent with $T$ iterations, and show that the gradient descent iterates fall inside local balls around either an initialization point or a reference point. Then we develop improved Rademacher complexity estimates by using the activation pattern of the ReLU function in these local balls. We apply our general result to NTK-separable data with a margin $\gamma$, and develop an almost optimal risk bound of the order $1/(n\gamma^2)$ for the ReLU network with a polylogarithmic width.

JMLR Optimization & Learning Theory · 发表于 2026,收录于 2026-09-10

Optimising Utility Functions in Multi-Objective Markov Decision Processes

Manel Rodriguez-Soto

摘要

Multi-Objective Markov Decision Processes (MOMDPs) are among the most prevalent formal frameworks for addressing sequential decision-making problems involving multiple, potentially conflicting objectives. In most MOMDP approaches, a utility function is employed to aggregate these objectives into a single scalar criterion that encodes user preferences. Despite its widespread adoption, the theoretical foundations of MOMDPs remain incomplete in two main respects: first, there is no general characterisation of the classes of utility functions that guarantee the existence of an optimal policy; second, we do not know which preference relations can be represented by utility functions. This work advances both lines of research through a theoretical analysis of MOMDPs. Specifically, we examine each problem under the two principal formulations of utility functions for MOMDPs: the Scalarised Expected Returns (SER) criterion and the Expected Scalarised Returns (ESR) criterion. Our formal findings allow us to derive formal conditions that describe the families of utility functions and preference relations for which MOMDP algorithms should focus. These analyses can guide the development of new MOMDP algorithms explicitly grounded in our formal results.

Econometrica Optimization & Learning Theory · 发表于 2026

Optimally‐Transported Generalized Method of Moments

Susanne Schennach, Vincent Starck

摘要

We propose a novel optimal transport‐based version of the Generalized Method of Moment (GMM). Instead of handling overidentification by reweighting the data to satisfy the moment conditions (as in Generalized Empirical Likelihood methods), this method proceeds by allowing for errors in the variables of the least mean‐square magnitude necessary to simultaneously satisfy all moment conditions. This approach, based on the notions of optimal transport and Wasserstein metric, aims to address the problem of assigning a logical interpretation to GMM results even when overidentification tests reject the null, a situation that cannot always be avoided in applications. We illustrate the method by revisiting Duranton, Morrow and Turner's (2014) study of the relationship between a city's exports and the extent of its transportation infrastructure. Our results corroborate theirs under weaker assumptions and provide insight into the error structure of the variables.

Econometrica Longitudinal Data · 发表于 2026

Optimal Shrinkage Estimation of Fixed Effects in Linear Panel Data Models

Soonwoo Kwon

摘要

Shrinkage methods are frequently used to improve the precision of least squares estimators of fixed effects. However, widely used shrinkage estimators guarantee improved precision only under strong distributional assumptions. I develop an estimator for the fixed effects that obtains the best possible mean squared error within a class of shrinkage estimators. This class includes conventional shrinkage estimators and the optimality does not require distributional assumptions. The estimator has an intuitive form and is easy to implement. Moreover, the fixed effects are allowed to vary with time and to be serially correlated, in which case the shrinkage optimally incorporates the underlying correlation structure. I also provide a method to forecast fixed effects one period ahead in this setting.

SiM Causal Inference Survival Analysis High-Dimensional · 发表于 2026,收录于 2026-08-17

Optimal Dynamic Treatment Regimes for High-Dimensional Accelerated Failure Time Model

Shijie Zhao, Wei Zhao

摘要

The increasing prevalence of high-dimensional covariates presents a significant challenge to precision medicine. To overcome this, we propose a novel multi-stage optimal dynamic treatment regime method specifically designed for high-dimensional accelerated failure time models. Our approach, which aims to maximize individual survival time, employs backward induction with counterfactual survival times as the optimization objective at each stage. Parameter estimation leverages weighted counterfactual survival times to account for censoring, followed by the slope determination by analysis of residuals (SDAR) algorithm. Theoretical analysis guarantees exponential convergence with non-asymptotic error bounds under mild conditions. Simulation studies demonstrate a substantial improvement in survival time estimation, particularly under high censoring rates, compared to existing methods. We illustrate the practical utility of our method by applying it to a sepsis dataset, revealing clinically meaningful treatment recommendations. *(Note: also touches survival/time-to-event modeling — filed here because the core methodological contribution is the dynamic treatment regime, not the survival model itself.)*

JMLR Other · 发表于 2026

Optimal Convergence Rates for Neural Operators

Mike Nguyen, Nicole Mücke

摘要

We introduce the neural tangent kernel (NTK) regime for two-layer neural operators and analyze their generalization properties. For early-stopped gradient descent (GD), we derive fast convergence rates that are known to be minimax optimal within the framework of non-parametric regression in reproducing kernel Hilbert spaces (RKHS). We provide bounds on the number of hidden neurons and the number of second-stage samples necessary for generalization. To justify our NTK regime, we additionally show that any operator approximable by a neural operator can also be approximated by an operator from the RKHS. A key application of neural operators is learning surrogate maps for the solution operators of partial differential equations (PDEs). We consider the standard Poisson equation to illustrate our theoretical findings with simulations.

JMLR Machine Learning Networks & Graphs Optimization & Learning Theory · 发表于 2026

Optimal Approximation and Generalization Errors for Deep Convolutional Neural Networks

Jinxin Wang, Shao-Bo Lin

摘要

This paper focuses on approximation and learning performances of deep convolutional neural networks with zero-padding and max-pooling. We prove that, to approximate $r$-smooth function, the approximation rates of deep convolutional neural networks with depth $L$ are of order $ (L/\log L)^{-2r/d} $, which is optimal up to a logarithmic factor. Furthermore, we deduce almost optimal generalization errors for implementing empirical risk minimization over deep convolutional neural networks. Our theoretical results are verified by several numerical experiments to show the power of the convolutional structure, zero-padding and max-pooling.

JMLR High-Dimensional · 发表于 2026

Online Generalized Sparse Regression: How Does Overparametrization Help?

Shuoguang Yang, Qiang Sun

摘要

Regularized sparse regression has been extensively studied in the offline setting, but online formulation remains relatively under-explored. This gap stems from four key challenges: (i) the infeasibility of dynamically updating the regularization parameter in every online round, (ii) managing storage and memory complexity, (iii) enabling real-time computation via closed-form updates rather than solving full optimization problems at each round, and (iv) achieving optimal statistical guarantees under realistic assumptions. In this paper, we propose an online generalized-sparsity-constrained regression framework, focusing on online cardinality-constrained linear regression and low-rank matrix sensing. Unlike online regularized regression, our constrained formulation eliminates the need for dynamic parameter tuning. We introduce an efficient online hard-thresholding algorithm that performs closed-form updates and requires storing only summary statistics, making it computationally, memory, and storage efficient. Despite the inherent nonconvexity and combinatorial nature of the formulation, our algorithm achieves global convergence at the optimal statistical rate under realistic assumptions, provided that the projection set is properly overparameterized. Numerical experiments demonstrate that our method consistently outperforms state-of-the-art alternatives.

JMLR High-Dimensional Machine Learning · 发表于 2026

Online Detection of Changes in Moment--Based Projections: When to Retrain Deep Learners or Update Portfolios?

Ansgar Steland

摘要

Training deep learning neural networks often requires massive amounts of computational ressources. We propose to sequentially monitor network predictions to trigger retraining only if the predictions are no longer valid. This can reduce drastically computational costs and opens a door to green deep learning. Our approach is based on the relationship to projected second moments monitoring, a problem also arising in other areas such as computational finance. Various open-end as well as closed-end monitoring rules are studied under mild assumptions on the training sample and the observations of the monitoring period. The results allow for high-dimensional non-stationary time series data and thus, especially, non-i.i.d. training data. Asymptotics is based on Gaussian approximations of projected partial sums allowing for an estimated projection vector. Estimation of projection vectors is studied both for classical non-$\ell_0$-sparsity as well as under sparsity. For the case that the optimal projection depends on the unknown covariance matrix, hard- and soft-thresholded estimators are studied. The method is analyzed by simulations and supported by synthetic data experiments.

JMLR Bayesian Methods · 发表于 2026

Online Bernstein-von Mises theorem

Jeyong Lee, Junhyeok Choi, Minwoo Chae

摘要

Online learning is an inferential paradigm in which parameters are updated incrementally from sequentially available data, in contrast to batch learning, where the entire dataset is processed at once. In this paper, we assume that mini-batches from the full dataset become available sequentially. The Bayesian framework, which updates beliefs about unknown parameters after observing each mini-batch, is naturally suited for online learning. At each step, we update the posterior distribution using the current prior and new observations, with the updated posterior serving as the prior for the next step. However, this recursive Bayesian updating is rarely computationally tractable unless the model and prior are conjugate. When the model is regular, the updated posterior can be approximated by a normal distribution, as justified by the Bernstein-von Mises theorem. We adopt a variational approximation at each step and investigate the frequentist properties of the final posterior obtained through this sequential procedure. Under mild assumptions, we show that the accumulated approximation error becomes negligible once the mini-batch size exceeds a threshold depending on the parameter dimension. As a result, the sequentially updated posterior is asymptotically indistinguishable from the full posterior.

JMLR Optimization & Learning Theory · 发表于 2026

On the Relevance of Byzantine Robust Optimization Against Data Poisoning

Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Rafael Pinot

摘要

The success of machine learning (ML) has been intimately linked with the availability of large amounts of data, typically collected from heterogeneous sources and processed on vast networks of computing devices (also called workers). Beyond accuracy, the use of ML in critical domains such as healthcare and autonomous driving calls for robustness against data poisoning and some faulty workers. The problem of Byzantine ML formalizes these robustness issues by considering a distributed ML environment in which workers (storing a portion of the global dataset) can deviate arbitrarily from the prescribed algorithm. Although the problem has attracted a lot of attention from a theoretical point of view, its practical importance for addressing realistic faults (where the behavior of any worker is locally constrained) remains unclear. It has been argued that the seemingly weaker threat model where only workers' local datasets get poisoned is more reasonable. We prove that, while tolerating a wider range of faulty behaviors, Byzantine ML yields solutions that are, in a precise sense, optimal even under the weaker data poisoning threat model. Then, we study a generic data poisoning model wherein some workers have fully-poisonous local data, i.e., their datasets are entirely corruptible, and the remainders have partially-poisonous local data, i.e., only a fraction of their local datasets is corruptible. We prove that Byzantine-robust schemes yield optimal solutions against both these forms of data poisoning, and that the former is more harmful when workers have heterogeneous local data.

JMLR Optimization & Learning Theory · 发表于 2026

On the Effectiveness of the z-Transform Method in Quadratic Optimization

Francis Bach

摘要

The z-transform of a sequence is a classical tool used in signal processing, control theory, computer science, and electrical engineering. It allows one to study sequences from their generating functions, with many operations that can be equivalently defined on the original sequence and its z-transform. In particular, the z-transform method focuses on asymptotic behaviors and allows the use of Taylor expansions. We present a sequence of results of increasing significance and difficulty for linear models and optimization algorithms, demonstrating the effectiveness and versatility of the z-transform method in deriving new asymptotic results. Starting from the simplest gradient descent iterations in an infinite-dimensional Hilbert space, we show how the spectral dimension characterizes the convergence behavior. We then extend the analysis to Nesterov acceleration, averaging techniques, and stochastic gradient descent.

JMLR Spatial & Time Series Machine Learning Nonparametric & Semiparametric · 发表于 2026

Nonparametric generative modeling for time series via Schrödinger bridge

Mohamed Hamdouche, Pierre Henry-Labordère, Huyên Pham

摘要

We propose a novel generative model for time series based on Schrödinger bridge (SB) approach. This consists in the entropic interpolation via optimal transport between a reference probability measure on path space and a target measure consistent with the joint data distribution of the time series. The solution is characterized by a stochastic differential equation on finite horizon with a path-dependent drift function, hence respec\-ting the temporal dynamics of the time series distribution. We estimate the drift function from data samples by nonparametric, e.g. kernel regression methods, and the simulation of the SB diffusion yields new synthetic data samples of the time series. The performance of our generative model is evaluated through a series of numerical experiments. First, we test with autoregressive models, a GARCH Model, and the example of fractional Brownian motion, and measure the accuracy of our algorithm with marginal, temporal dependencies metrics, and predictive scores. Next, we use our SB generated synthetic samples for the application to deep hedging on real-data sets.

JMLR Nonparametric & Semiparametric · 发表于 2026

Nonparametric Spectral Density Estimation using Interactive Mechanisms under Local Differential Privacy

Cristina Butucea, Karolina Klockmann, Tatyana Krivobokova

摘要

We study the problem of estimating the spectral density of a centered stationary Gaussian time series under local differential privacy constraints. Specifically, we propose new interactive privacy mechanisms for three tasks: recovering a single covariance coefficient, recovering the spectral density at a fixed frequency, and global recovery. Our approach achieves faster rates through a two-stage process: we first apply the Laplace mechanism to the truncated value, and then use the resulting privatized sample to learn about the dependence mechanism in the time series. For spectral densities belonging to Hölder and Sobolev smoothness classes, we demonstrate that our algorithms improve upon the non-interactive mechanism of Kroll (2024) for small privacy parameter $\alpha$, since the pointwise rates depend on $n\alpha^2$ instead of $n\alpha^4$. Moreover, we show that the rate $(n\alpha^4)^{-1}$ is optimal for estimating a covariance coefficient with non-interactive mechanisms. However, the $L_2$ rate of our interactive estimator is slower than the pointwise rate. We show how to use these procedures to provide a bona fide locally differentially private estimator of the entire covariance matrix. A simulation study validates our findings.

JMLR High-Dimensional Nonparametric & Semiparametric Latent Variables & Mixtures · 发表于 2026

Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies

Sébastien Lachapelle, Pau Rodríguez López, Yash Sharma, Katie Everett, Rémi Le Priol, Alexandre Lacoste, Simon Lacoste-Julien

摘要

This work introduces a novel principle for disentanglement we call mechanism sparsity regularization, which applies when the latent factors of interest depend sparsely on observed auxiliary variables and/or past latent factors. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors and the sparse causal graphical model that explains them. We develop a nonparametric identifiability theory that formalizes this principle and shows that the latent factors can be recovered by regularizing the learned causal graph to be sparse, under some assumptions such as the absence of instantaneous causal effects between latent factors. More precisely, we show identifiability up to a novel equivalence relation we call consistency, which allows some latent factors to remain entangled (hence the term partial disentanglement). To describe the structure of this entanglement, we introduce the notions of entanglement graphs and graph preserving functions. We further provide a graphical criterion which guarantees complete disentanglement, that is identifiability up to permutations and element-wise transformations. We demonstrate the scope of the mechanism sparsity principle as well as the assumptions it relies on with several worked out examples. For instance, the framework shows how one can leverage multi-node interventions with unknown targets on the latent factors to disentangle them. We further draw connections between our nonparametric results and the now popular exponential family assumption. Lastly, we propose an estimation procedure based on variational autoencoders and a sparsity constraint and demonstrate it on various synthetic datasets. This work is meant to be a significantly extended version of a work published at CLeaR 2022.

第 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 页