Statistics Theory
New submissions
[ showing up to 2000 entries per page: fewer  more ]
New submissions for Wed, 11 Dec 19
 [1] arXiv:1912.04629 [pdf, ps, other]

Title: Classification under local differential privacyComments: 12 pagesSubjects: Statistics Theory (math.ST); Methodology (stat.ME); Machine Learning (stat.ML)
We consider the binary classification problem in a setup that preserves the privacy of the original sample. We provide a privacy mechanism that is locally differentially private and then construct a classifier based on the private sample that is universally consistent in Euclidean spaces. Under stronger assumptions, we establish the minimax rates of convergence of the excess risk and see that they are slower than in the case when the original sample is available.
 [2] arXiv:1912.04677 [pdf, other]

Title: Testing and Estimating ChangePoints in the Covariance Matrix of a HighDimensional Time SeriesAuthors: Ansgar StelandSubjects: Statistics Theory (math.ST); Probability (math.PR); Applications (stat.AP)
This paper studies methods for testing and estimating changepoints in the covariance structure of a highdimensional linear time series. The assumed framework allows for a large class of multivariate linear processes (including vector autoregressive moving average (VARMA) models) of growing dimension and spiked covariance models. The approach uses bilinear forms of the centered or noncentered sample variancecovariance matrix. Changepoint testing and estimation are based on maximally selected weighted cumulated sum (CUSUM) statistics. Large sample approximations under a changepoint regime are provided including a multivariate CUSUM transform of increasing dimension. For the unknown asymptotic variance and covariance parameters associated to (pairs of) CUSUM statistics we propose consistent estimators. Based on weak laws of large numbers for their sequential versions, we also consider stopped sample estimation where observations until the estimated changepoint are used. Finite sample properties of the procedures are investigated by simulations and their application is illustrated by analyzing a real data set from environmetrics.
 [3] arXiv:1912.04869 [pdf, other]

Title: Adaptive Manifold ClusteringSubjects: Statistics Theory (math.ST)
We extend the theoretical study of a recently proposed nonparametric clustering algorithm called Adaptive Weights Clustering (AWC). In particular, we are interested in the case of highdimensional data lying in the vicinity of a lowerdimensional nonlinear submanifold with positive reach. After a slight adjustment and under rather general assumptions for the cluster structure, the algorithm turns out to be nearly optimal in detecting local inhomogeneities, while aggregating homogeneous data with a high probability. We also adress the problem of parameter tuning.
Crosslists for Wed, 11 Dec 19
 [4] arXiv:1912.04533 (crosslist from cs.LG) [pdf, other]

Title: Exact expressions for double descent and implicit regularization via surrogate random designSubjects: Machine Learning (cs.LG); Statistics Theory (math.ST); Machine Learning (stat.ML)
Double descent refers to the phase transition that is exhibited by the generalization error of unregularized learning models when varying the ratio between the number of parameters and the number of training samples. The recent success of highly overparameterized machine learning models such as deep neural networks has motivated a theoretical analysis of the double descent phenomenon in classical models such as linear regression which can also generalize well in the overparameterized regime. We build on recent advances in Randomized Numerical Linear Algebra (RandNLA) to provide the first exact nonasymptotic expressions for double descent of the minimum norm linear estimator. Our approach involves constructing what we call a surrogate random design to replace the standard i.i.d. design of the training sample. This surrogate design admits exact expressions for the mean squared error of the estimator while preserving the key properties of the standard design. We also establish an exact implicit regularization result for overparameterized training samples. In particular, we show that, for the surrogate design, the implicit bias of the unregularized minimum norm estimator precisely corresponds to solving a ridgeregularized least squares problem on the population distribution.
 [5] arXiv:1912.04738 (crosslist from stat.ML) [pdf, other]

Title: Histogram Transform Ensembles for Largescale RegressionComments: arXiv admin note: text overlap with arXiv:1911.11581Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
We propose a novel algorithm for largescale regression problems named histogram transform ensembles (HTE), composed of random rotations, stretchings, and translations. First of all, we investigate the theoretical properties of HTE when the regression function lies in the H\"{o}lder space $C^{k,\alpha}$, $k \in \mathbb{N}_0$, $\alpha \in (0,1]$. In the case that $k=0, 1$, we adopt the constant regressors and develop the na\"{i}ve histogram transforms (NHT). Within the space $C^{0,\alpha}$, although almost optimal convergence rates can be derived for both single and ensemble NHT, we fail to show the benefits of ensembles over single estimators theoretically. In contrast, in the subspace $C^{1,\alpha}$, we prove that if $d \geq 2(1+\alpha)/\alpha$, the lower bound of the convergence rates for single NHT turns out to be worse than the upper bound of the convergence rates for ensemble NHT. In the other case when $k \geq 2$, the NHT may no longer be appropriate in predicting smoother regression functions. Instead, we apply kernel histogram transforms (KHT) equipped with smoother regressors such as support vector machines (SVMs), and it turns out that both single and ensemble KHT enjoy almost optimal convergence rates. Then we validate the above theoretical results by numerical experiments. On the one hand, simulations are conducted to elucidate that ensemble NHT outperform single NHT. On the other hand, the effects of bin sizes on accuracy of both NHT and KHT also accord with theoretical analysis. Last but not least, in the realdata experiments, comparisons between the ensemble KHT, equipped with adaptive histogram transforms, and other stateoftheart largescale regression estimators verify the effectiveness and accuracy of our algorithm.
 [6] arXiv:1912.04858 (crosslist from math.PR) [pdf, ps, other]

Title: Rates of convergence to the local time of Oscillating and Skew Brownian MotionsAuthors: Sara MazzonettoSubjects: Probability (math.PR); Statistics Theory (math.ST)
In this paper a class of statistics based on high frequency observations of oscillating Brownian motions and skew Brownian motions is considered. Their convergence rate towards the local time of the underling process is obtained in form of a Central Limit Theorem.
Replacements for Wed, 11 Dec 19
 [7] arXiv:1907.09617 (replaced) [pdf, other]

Title: Hierarchical Transformed Scale Mixtures for Flexible Modeling of Spatial Extremes on Datasets with Many LocationsSubjects: Statistics Theory (math.ST); Methodology (stat.ME)
 [8] arXiv:1909.10024 (replaced) [pdf, other]

Title: Distributionfree consistent independence tests via Hallin's multivariate rankComments: In this (3rd) version, we added more referencesSubjects: Statistics Theory (math.ST)
 [9] arXiv:1910.08520 (replaced) [pdf, other]

Title: Optimization Hierarchy for Fair Statistical Decision ProblemsSubjects: Statistics Theory (math.ST); Optimization and Control (math.OC)
 [10] arXiv:1904.03920 (replaced) [pdf, other]

Title: A Generalization Bound for Online Variational InferenceComments: Published in the proceedings of ACML 2019Journalref: Proceedings in Machine Learning Research, 2019, vol. 101, pp. 662677Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST); Computation (stat.CO)
 [11] arXiv:1906.10075 (replaced) [pdf, other]

Title: DistributionIndependent PAC Learning of Halfspaces with Massart NoiseSubjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Statistics Theory (math.ST); Machine Learning (stat.ML)
 [12] arXiv:1912.04151 (replaced) [pdf, other]

Title: Identification of causal intervention effects under contagionSubjects: Applications (stat.AP); Statistics Theory (math.ST); Populations and Evolution (qbio.PE)
[ showing up to 2000 entries per page: fewer  more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, stat, recent, 1912, contact, help (Access key information)