We gratefully acknowledge support from
the Simons Foundation and member institutions.

Statistics Theory

New submissions

[ total of 12 entries: 1-12 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Wed, 11 Dec 19

[1]  arXiv:1912.04629 [pdf, ps, other]
Title: Classification under local differential privacy
Comments: 12 pages
Subjects: Statistics Theory (math.ST); Methodology (stat.ME); Machine Learning (stat.ML)

We consider the binary classification problem in a setup that preserves the privacy of the original sample. We provide a privacy mechanism that is locally differentially private and then construct a classifier based on the private sample that is universally consistent in Euclidean spaces. Under stronger assumptions, we establish the minimax rates of convergence of the excess risk and see that they are slower than in the case when the original sample is available.

[2]  arXiv:1912.04677 [pdf, other]
Title: Testing and Estimating Change-Points in the Covariance Matrix of a High-Dimensional Time Series
Authors: Ansgar Steland
Subjects: Statistics Theory (math.ST); Probability (math.PR); Applications (stat.AP)

This paper studies methods for testing and estimating change-points in the covariance structure of a high-dimensional linear time series. The assumed framework allows for a large class of multivariate linear processes (including vector autoregressive moving average (VARMA) models) of growing dimension and spiked covariance models. The approach uses bilinear forms of the centered or non-centered sample variance-covariance matrix. Change-point testing and estimation are based on maximally selected weighted cumulated sum (CUSUM) statistics. Large sample approximations under a change-point regime are provided including a multivariate CUSUM transform of increasing dimension. For the unknown asymptotic variance and covariance parameters associated to (pairs of) CUSUM statistics we propose consistent estimators. Based on weak laws of large numbers for their sequential versions, we also consider stopped sample estimation where observations until the estimated change-point are used. Finite sample properties of the procedures are investigated by simulations and their application is illustrated by analyzing a real data set from environmetrics.

[3]  arXiv:1912.04869 [pdf, other]
Title: Adaptive Manifold Clustering
Subjects: Statistics Theory (math.ST)

We extend the theoretical study of a recently proposed nonparametric clustering algorithm called Adaptive Weights Clustering (AWC). In particular, we are interested in the case of high-dimensional data lying in the vicinity of a lower-dimensional non-linear submanifold with positive reach. After a slight adjustment and under rather general assumptions for the cluster structure, the algorithm turns out to be nearly optimal in detecting local inhomogeneities, while aggregating homogeneous data with a high probability. We also adress the problem of parameter tuning.

Cross-lists for Wed, 11 Dec 19

[4]  arXiv:1912.04533 (cross-list from cs.LG) [pdf, other]
Title: Exact expressions for double descent and implicit regularization via surrogate random design
Subjects: Machine Learning (cs.LG); Statistics Theory (math.ST); Machine Learning (stat.ML)

Double descent refers to the phase transition that is exhibited by the generalization error of unregularized learning models when varying the ratio between the number of parameters and the number of training samples. The recent success of highly over-parameterized machine learning models such as deep neural networks has motivated a theoretical analysis of the double descent phenomenon in classical models such as linear regression which can also generalize well in the over-parameterized regime. We build on recent advances in Randomized Numerical Linear Algebra (RandNLA) to provide the first exact non-asymptotic expressions for double descent of the minimum norm linear estimator. Our approach involves constructing what we call a surrogate random design to replace the standard i.i.d. design of the training sample. This surrogate design admits exact expressions for the mean squared error of the estimator while preserving the key properties of the standard design. We also establish an exact implicit regularization result for over-parameterized training samples. In particular, we show that, for the surrogate design, the implicit bias of the unregularized minimum norm estimator precisely corresponds to solving a ridge-regularized least squares problem on the population distribution.

[5]  arXiv:1912.04738 (cross-list from stat.ML) [pdf, other]
Title: Histogram Transform Ensembles for Large-scale Regression
Comments: arXiv admin note: text overlap with arXiv:1911.11581
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)

We propose a novel algorithm for large-scale regression problems named histogram transform ensembles (HTE), composed of random rotations, stretchings, and translations. First of all, we investigate the theoretical properties of HTE when the regression function lies in the H\"{o}lder space $C^{k,\alpha}$, $k \in \mathbb{N}_0$, $\alpha \in (0,1]$. In the case that $k=0, 1$, we adopt the constant regressors and develop the na\"{i}ve histogram transforms (NHT). Within the space $C^{0,\alpha}$, although almost optimal convergence rates can be derived for both single and ensemble NHT, we fail to show the benefits of ensembles over single estimators theoretically. In contrast, in the subspace $C^{1,\alpha}$, we prove that if $d \geq 2(1+\alpha)/\alpha$, the lower bound of the convergence rates for single NHT turns out to be worse than the upper bound of the convergence rates for ensemble NHT. In the other case when $k \geq 2$, the NHT may no longer be appropriate in predicting smoother regression functions. Instead, we apply kernel histogram transforms (KHT) equipped with smoother regressors such as support vector machines (SVMs), and it turns out that both single and ensemble KHT enjoy almost optimal convergence rates. Then we validate the above theoretical results by numerical experiments. On the one hand, simulations are conducted to elucidate that ensemble NHT outperform single NHT. On the other hand, the effects of bin sizes on accuracy of both NHT and KHT also accord with theoretical analysis. Last but not least, in the real-data experiments, comparisons between the ensemble KHT, equipped with adaptive histogram transforms, and other state-of-the-art large-scale regression estimators verify the effectiveness and accuracy of our algorithm.

[6]  arXiv:1912.04858 (cross-list from math.PR) [pdf, ps, other]
Title: Rates of convergence to the local time of Oscillating and Skew Brownian Motions
Authors: Sara Mazzonetto
Subjects: Probability (math.PR); Statistics Theory (math.ST)

In this paper a class of statistics based on high frequency observations of oscillating Brownian motions and skew Brownian motions is considered. Their convergence rate towards the local time of the underling process is obtained in form of a Central Limit Theorem.

Replacements for Wed, 11 Dec 19

[7]  arXiv:1907.09617 (replaced) [pdf, other]
Title: Hierarchical Transformed Scale Mixtures for Flexible Modeling of Spatial Extremes on Datasets with Many Locations
Subjects: Statistics Theory (math.ST); Methodology (stat.ME)
[8]  arXiv:1909.10024 (replaced) [pdf, other]
Title: Distribution-free consistent independence tests via Hallin's multivariate rank
Comments: In this (3rd) version, we added more references
Subjects: Statistics Theory (math.ST)
[9]  arXiv:1910.08520 (replaced) [pdf, other]
Title: Optimization Hierarchy for Fair Statistical Decision Problems
Subjects: Statistics Theory (math.ST); Optimization and Control (math.OC)
[10]  arXiv:1904.03920 (replaced) [pdf, other]
Title: A Generalization Bound for Online Variational Inference
Comments: Published in the proceedings of ACML 2019
Journal-ref: Proceedings in Machine Learning Research, 2019, vol. 101, pp. 662-677
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST); Computation (stat.CO)
[11]  arXiv:1906.10075 (replaced) [pdf, other]
Title: Distribution-Independent PAC Learning of Halfspaces with Massart Noise
Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Statistics Theory (math.ST); Machine Learning (stat.ML)
[12]  arXiv:1912.04151 (replaced) [pdf, other]
Title: Identification of causal intervention effects under contagion
Subjects: Applications (stat.AP); Statistics Theory (math.ST); Populations and Evolution (q-bio.PE)
[ total of 12 entries: 1-12 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, stat, recent, 1912, contact, help  (Access key information)