Data Structures and Algorithms
New submissions
[ showing up to 2000 entries per page: fewer  more ]
New submissions for Fri, 18 Jun 21
 [1] arXiv:2106.09350 [pdf, other]

Title: Identifiability of AMP chain graph modelsComments: 16 pages, 4 figuresSubjects: Data Structures and Algorithms (cs.DS); Machine Learning (stat.ML)
We study identifiability of AnderssonMadiganPerlman (AMP) chain graph models, which are a common generalization of linear structural equation models and Gaussian graphical models. AMP models are described by DAGs on chain components which themselves are undirected graphs.
For a known chain component decomposition, we show that the DAG on the chain components is identifiable if the determinants of the residual covariance matrices of the chain components are monotone nondecreasing in topological order. This condition extends the equal variance identifiability criterion for Bayes nets, and it can be generalized from determinants to any superadditive function on positive semidefinite matrices. When the component decomposition is unknown, we describe conditions that allow recovery of the full structure using a polynomial time algorithm based on submodular function minimization. We also conduct experiments comparing our algorithm's performance against existing baselines.  [2] arXiv:2106.09689 [pdf, ps, other]

Title: Statistical Query Lower Bounds for ListDecodable Linear RegressionSubjects: Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG); Statistics Theory (math.ST); Machine Learning (stat.ML)
We study the problem of listdecodable linear regression, where an adversary can corrupt a majority of the examples. Specifically, we are given a set $T$ of labeled examples $(x, y) \in \mathbb{R}^d \times \mathbb{R}$ and a parameter $0< \alpha <1/2$ such that an $\alpha$fraction of the points in $T$ are i.i.d. samples from a linear regression model with Gaussian covariates, and the remaining $(1\alpha)$fraction of the points are drawn from an arbitrary noise distribution. The goal is to output a small list of hypothesis vectors such that at least one of them is close to the target regression vector. Our main result is a Statistical Query (SQ) lower bound of $d^{\mathrm{poly}(1/\alpha)}$ for this problem. Our SQ lower bound qualitatively matches the performance of previously developed algorithms, providing evidence that current upper bounds for this task are nearly best possible.
Crosslists for Fri, 18 Jun 21
 [3] arXiv:2106.09207 (crosslist from cs.LG) [pdf, other]

Title: On the Power of Preconditioning in Sparse Linear RegressionComments: 73 pages, 5 figuresSubjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Statistics Theory (math.ST); Machine Learning (stat.ML)
Sparse linear regression is a fundamental problem in highdimensional statistics, but strikingly little is known about how to efficiently solve it without restrictive conditions on the design matrix. We consider the (correlated) random design setting, where the covariates are independently drawn from a multivariate Gaussian $N(0,\Sigma)$ with $\Sigma : n \times n$, and seek estimators $\hat{w}$ minimizing $(\hat{w}w^*)^T\Sigma(\hat{w}w^*)$, where $w^*$ is the $k$sparse ground truth. Information theoretically, one can achieve strong error bounds with $O(k \log n)$ samples for arbitrary $\Sigma$ and $w^*$; however, no efficient algorithms are known to match these guarantees even with $o(n)$ samples, without further assumptions on $\Sigma$ or $w^*$. As far as hardness, computational lower bounds are only known with worstcase design matrices. Randomdesign instances are known which are hard for the Lasso, but these instances can generally be solved by Lasso after a simple changeofbasis (i.e. preconditioning).
In this work, we give upper and lower bounds clarifying the power of preconditioning in sparse linear regression. First, we show that the preconditioned Lasso can solve a large class of sparse linear regression problems nearly optimally: it succeeds whenever the dependency structure of the covariates, in the sense of the Markov property, has low treewidth  even if $\Sigma$ is highly illconditioned. Second, we construct (for the first time) randomdesign instances which are provably hard for an optimally preconditioned Lasso. In fact, we complete our treewidth classification by proving that for any treewidth$t$ graph, there exists a Gaussian Markov Random Field on this graph such that the preconditioned Lasso, with any choice of preconditioner, requires $\Omega(t^{1/20})$ samples to recover $O(\log n)$sparse signals when covariates are drawn from this model.  [4] arXiv:2106.09363 (crosslist from condmat.mtrlsci) [pdf, ps, other]

Title: Similarity of particle systems using an invariant root mean square deviation measureSubjects: Materials Science (condmat.mtrlsci); Data Structures and Algorithms (cs.DS)
Determining whether two particle systems are similar is a common problem in particle simulations. When the comparison should be invariant under permutations, orthogonal transformations, and translations of the systems, special techniques are needed. We present an algorithm that can test particle systems of finite size for similarity and, if they are similar, can find the optimal alignment between them. Our approach is based on an invariant version of the root mean square deviation (RMSD) measure and is capable of finding the globally optimal solution in $O(n^3)$ operations where $n$ is the number of threedimensional particles.
 [5] arXiv:2106.09481 (crosslist from math.OC) [pdf, other]

Title: Stochastic BiasReduced Gradient MethodsSubjects: Optimization and Control (math.OC); Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG)
We develop a new primitive for stochastic optimization: a lowbias, lowcost estimator of the minimizer $x_\star$ of any Lipschitz stronglyconvex function. In particular, we use a multilevel MonteCarlo approach due to Blanchet and Glynn to turn any optimal stochastic gradient method into an estimator of $x_\star$ with bias $\delta$, variance $O(\log(1/\delta))$, and an expected sampling cost of $O(\log(1/\delta))$ stochastic gradient evaluations. As an immediate consequence, we obtain cheap and nearly unbiased gradient estimators for the MoreauYoshida envelope of any Lipschitz convex function, allowing us to perform dimensionfree randomized smoothing.
We demonstrate the potential of our estimator through four applications. First, we develop a method for minimizing the maximum of $N$ functions, improving on recent results and matching a lower bound up logarithmic factors. Second and third, we recover stateoftheart rates for projectionefficient and gradientefficient optimization using simple algorithms with a transparent analysis. Finally, we show that an improved version of our estimator would yield a nearly lineartime, optimalutility, differentiallyprivate nonsmooth stochastic optimization method.  [6] arXiv:2106.09663 (crosslist from math.OC) [pdf, ps, other]

Title: A Short Note of PAGE: Optimal Convergence Rates for Nonconvex OptimizationAuthors: Zhize LiComments: 4 pagesSubjects: Optimization and Control (math.OC); Artificial Intelligence (cs.AI); Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG); Machine Learning (stat.ML)
In this note, we first recall the nonconvex problem setting and introduce the optimal PAGE algorithm (Li et al., ICML'21). Then we provide a simple and clean convergence analysis of PAGE for achieving optimal convergence rates. Moreover, PAGE and its analysis can be easily adopted and generalized to other works. We hope that this note provides the insights and is helpful for future works.
Replacements for Fri, 18 Jun 21
 [7] arXiv:1807.11702 (replaced) [pdf, ps, other]

Title: Efficient Computation of Sequence MappabilityAuthors: Panagiotis Charalampopoulos, Costas S. Iliopoulos, Tomasz Kociumaka, Solon P. Pissis, Jakub Radoszewski, Juliusz StraszyńskiComments: Accepted to SPIRE 2018Subjects: Data Structures and Algorithms (cs.DS)
 [8] arXiv:2106.06525 (replaced) [pdf, other]

Title: ExtendedHyperLogLog: Analysis of a new Cardinality EstimatorAuthors: Tal OhayonSubjects: Data Structures and Algorithms (cs.DS)
 [9] arXiv:2106.02129 (replaced) [pdf, ps, other]

Title: The Algorithmic Phase Transition of Random $k$SAT for Low Degree PolynomialsComments: 44 pages, added referencesSubjects: Computational Complexity (cs.CC); Data Structures and Algorithms (cs.DS); Mathematical Physics (mathph); Probability (math.PR); Machine Learning (stat.ML)
 [10] arXiv:2106.08652 (replaced) [pdf, other]

Title: MaxminFair Ranking: Individual Fairness under GroupFairness ConstraintsComments: In proceedings of KDD 2021Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Data Structures and Algorithms (cs.DS)
[ showing up to 2000 entries per page: fewer  more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, cs, recent, 2106, contact, help (Access key information)