Optimization and Control
New submissions
[ showing up to 2000 entries per page: fewer  more ]
New submissions for Fri, 17 Jan 20
 [1] arXiv:2001.05537 [pdf, other]

Title: Accelerated DualAveraging PrimalDual Method for Composite Convex MinimizationJournalref: Optimization Methods and Software 2020Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Machine Learning (stat.ML)
Dual averagingtype methods are widely used in industrial machine learning applications due to their ability to promoting solution structure (e.g., sparsity) efficiently. In this paper, we propose a novel accelerated dualaveraging primaldual algorithm for minimizing a composite convex function. We also derive a stochastic version of the proposed method which solves empirical risk minimization, and its advantages on handling sparse data are demonstrated both theoretically and empirically.
 [2] arXiv:2001.05739 [pdf, other]

Title: Centering ADMM for the Semidefinite Relaxation of the QAPSubjects: Optimization and Control (math.OC)
We propose a new method for solving the semidefinite (SD) relaxation of the quadratic assignment problem (QAP), called the Centering ADMM. The Centering ADMM is an alternating direction method of multipliers (ADMM) combining the centering steps used in the interiorpoint method. The first stage of the Centering ADMM updates the iterate so that it approaches the central path by incorporating a barrier function term into the objective function, as in the interiorpoint method. If the current iterate is sufficiently close to the central path with a sufficiently small value of the barrier parameter, the method switches to the Standard ADMM. To observe the effect of the centering steps, we conducted numerical experiments with SD relaxation problems of instances in the QAPLIB. The results demonstrate that the centering steps are quite efficient for some classes of instances.
 [3] arXiv:2001.05740 [pdf, other]

Title: Lifting to Passivity for $\mathcal{H}_2$GainScheduling Synthesis with Full Block ScalingsSubjects: Optimization and Control (math.OC)
We focus on the $\mathcal{H}_2$gainscheduling synthesis problem for timevarying parametric scheduling blocks with full block scalings. Recently, we have presented a solution of this problem for $D$ and positive real scalings by relying on a convexifying transformation for the controller parameters and by guaranteeing finiteness of the $\mathcal{H}_2$norm for the closedloop system with suitable linear fractional plant and controller representations. We extend these methods to full block scalings by designing a triangular scheduling function and by introducing a new lifting technique for gainscheduled synthesis that enables convexification.
 [4] arXiv:2001.05768 [pdf, ps, other]

Title: Some convergent results for Backtracking Gradient Descent method on Banach spacesAuthors: Tuyen Trung TruongComments: 8 pagesSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Analysis of PDEs (math.AP); Functional Analysis (math.FA); Machine Learning (stat.ML)
Our main result concerns the following condition:
{\bf Condition C.} Let $X$ be a Banach space. A $C^1$ function $f:X\rightarrow \mathbb{R}$ satisfies Condition C if whenever $\{x_n\}$ weakly converges to $x$ and $\lim _{n\rightarrow\infty}\nabla f(x_n)=0$, then $\nabla f(x)=0$.
We assume that there is given a canonical isomorphism between $X$ and its dual $X^*$, for example when $X$ is a Hilbert space.
{\bf Theorem.} Let $X$ be a reflexive, complete Banach space and $f:X\rightarrow \mathbb{R}$ be a $C^2$ function which satisfies Condition C. Moreover, we assume that for every bounded set $S\subset X$, then $\sup _{x\in S}\nabla ^2f(x)<\infty$. We choose a random point $x_0\in X$ and construct by the Local Backtracking GD procedure (which depends on $3$ hyperparameters $\alpha ,\beta ,\delta _0$, see later for details) the sequence $x_{n+1}=x_n\delta (x_n)\nabla f(x_n)$. Then we have:
1) Every cluster point of $\{x_n\}$, in the {\bf weak} topology, is a critical point of $f$.
2) Either $\lim _{n\rightarrow\infty}f(x_n)=\infty$ or $\lim _{n\rightarrow\infty}x_{n+1}x_n=0$.
3) Here we work with the weak topology. Let $\mathcal{C}$ be the set of critical points of $f$. Assume that $\mathcal{C}$ has a bounded component $A$. Let $\mathcal{B}$ be the set of cluster points of $\{x_n\}$. If $\mathcal{B}\cap A\not= \emptyset$, then $\mathcal{B}\subset A$ and $\mathcal{B}$ is connected.
4) Assume that $f$ has at most countably many saddle points. Then for generic choices of $\alpha ,\beta ,\delta _0$ and the initial point $x_0$, if the sequence $\{x_n\}$ converges  in the {\bf weak} topology, then the limit point cannot be a saddle point.  [5] arXiv:2001.05795 [pdf, other]

Title: Stable and Robust LQR Design via Scenario ApproachComments: 14 pages, 3 figuresSubjects: Optimization and Control (math.OC)
Linear Quadratic Regulator (LQR) design is one of the most classical optimal control problems, whose wellknown solution is an input sequence expressed as a statefeedback. In this work, finitehorizon and discretetime LQR is solved under stability constraints and uncertain system dynamics. The resulting feedback controller balances cost value and closedloop stability. Robustness of the solution is modeled using the scenario approach, without requiring any probabilistic description of the uncertainty in the system matrices. The new methods are tested and compared on the Leslie growth model, where we control population size while minimizing a suitable finitehorizon cost function.
 [6] arXiv:2001.05915 [pdf, ps, other]

Title: Adaptive iterative singular value thresholding algorithm to lowrank matrix recoverySubjects: Optimization and Control (math.OC)
The problem of recovering a lowrank matrix from the linear constraints, known as affine matrix rank minimization problem, has been attracting extensive attention in recent years. In general, affine matrix rank minimization problem is a NPhard. In our latest work, a nonconvex fraction function is studied to approximate the rank function in affine matrix rank minimization problem and translate the NPhard affine matrix rank minimization problem into a transformed affine matrix rank minimization problem. A scheme of iterative singular value thresholding algorithm is generated to solve the regularization transformed affine matrix rank minimization problem. However, one of the drawbacks for our iterative singular value thresholding algorithm is that the parameter $a$, which influences the behaviour of nonconvex fraction function in the regularization transformed affine matrix rank minimization problem, needs to be determined manually in every simulation. In fact, how to determine the optimal parameter $a$ is not an easy problem. Here instead, in this paper, we will generate an adaptive iterative singular value thresholding algorithm to solve regularization transformed affine matrix rank minimization problem. When doing so, our new algorithm will be intelligent both for the choice of the regularization parameter $\lambda$ and the parameter $a$.
Crosslists for Fri, 17 Jan 20
 [7] arXiv:2001.05530 (crosslist from math.NA) [pdf, other]

Title: Biorthogonal greedy algorithms in convex optimizationSubjects: Numerical Analysis (math.NA); Optimization and Control (math.OC)
The study of greedy approximation in the context of convex optimization is becoming a promising research direction as greedy algorithms are actively being employed to construct sparse minimizers for convex functions with respect to given sets of elements. In this paper we propose a unified way of analyzing a certain kind of greedytype algorithms for the minimization of convex functions on Banach spaces. Specifically, we define the class of Weak Biorthogonal Greedy Algorithms for convex optimization that contains a wide range of greedy algorithms. We analyze the introduced class of algorithms and establish the properties of convergence, rate of convergence, and numerical stability, which is understood in the sense that the steps of the algorithm are allowed to be performed not precisely but with controlled computational inaccuracies. We show that the following wellknown algorithms for convex optimization  the Weak Chebyshev Greedy Algorithm (co) and the Weak Greedy Algorithm with Free Relaxation (co)  belong to this class, and introduce a new algorithm  the Rescaled Weak Relaxed Greedy Algorithm (co). Presented numerical experiments demonstrate the practical performance of the aforementioned greedy algorithms in the setting of convex minimization as compared to optimization with regularization, which is the conventional approach of constructing sparse minimizers.
 [8] arXiv:2001.05781 (crosslist from math.NA) [pdf, ps, other]

Title: Optimal parameter for the SORlike iteration method for solving the system of absolute value equationsComments: 15 pages, 5 figures, 6 tablesSubjects: Numerical Analysis (math.NA); Optimization and Control (math.OC)
The absolute value equations (AVE) $Ax  x  b = 0$ is of interest of the optimization community. Recently, the SORlike iteration method has been developed (Ke and Ma [{\em Appl. Math. Comput.}, 311:195202, 2017]) and shown to be efficient for numerically solving the AVE with $\nu=\A^{1}\_2<1$ (Ke and Ma [{\em Appl. Math. Comput.}, 311:195202, 2017]; Guo, Wu and Li [{\em Appl. Math. Lett.}, 97:107113, 2019]). Since the SORlike iteration method is oneparameterdependent, it is an important problem to determine the optimal iteration parameter. In this paper, we revisit the convergence conditions of the SORlike iteration method proposed by Ke and Ma ([{\em Appl. Math. Comput.}, 311:195202, 2017]). Furthermore, we explore the optimal parameter which minimizes $\T(\omega)\_2$ and the approximate optimal parameter which minimizes $\eta=\max\{1\omega,\nu\omega^2\}$. The optimal and approximate optimal parameters are iterationindependent. Numerical results demonstrate that the SORlike iteration method with the optimal parameter is superior to that with the approximate optimal parameter proposed by Guo, Wu and Li ([{\em Appl. Math. Lett.}, 97:107113, 2019]).
 [9] arXiv:2001.05820 (crosslist from math.CO) [pdf, other]

Title: Probabilistic values for simplicial complexesAuthors: Ivan MartinoComments: 23 pages, 1 figureSubjects: Combinatorics (math.CO); Optimization and Control (math.OC)
In this manuscript, we define and study probabilistic values for cooperative games on simplicial complexes. Inspired by the work of Weber "Probabilistic values for games", we establish the new theory step by step, following the classical axiomatization, i.e. using the linearity axiom, the dummy axiom, etc.
Furthermore, we define Shapley values on simplicial complexes generalizing the classical notion in literature. Remarkably, the traditional axiomatization of Shapley values can be extended to this general setting for a rather interesting class of complexes that generalize the notion of vertextransitive graphs and vertexhomogeneous simplicial complexes. These combinatorial objects are very popular in the literature because of the study of Evasiveness Conjecture in Complexity Theory.  [10] arXiv:2001.05992 (crosslist from cs.LG) [pdf, other]

Title: Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear NetworksComments: International Conference on Learning Representations (ICLR) 2020Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Optimization and Control (math.OC); Machine Learning (stat.ML)
The selection of initial parameter values for gradientbased optimization of deep neural networks is one of the most impactful hyperparameter choices in deep learning systems, affecting both convergence times and model performance. Yet despite significant empirical and theoretical analysis, relatively little has been proved about the concrete effects of different initialization schemes. In this work, we analyze the effect of initialization in deep linear networks, and provide for the first time a rigorous proof that drawing the initial weights from the orthogonal group speeds up convergence relative to the standard Gaussian initialization with iid weights. We show that for deep networks, the width needed for efficient convergence to a global minimum with orthogonal initializations is independent of the depth, whereas the width needed for efficient convergence with Gaussian initializations scales linearly in the depth. Our results demonstrate how the benefits of a good initialization can persist throughout learning, suggesting an explanation for the recent empirical successes found by initializing very deep nonlinear networks according to the principle of dynamical isometry.
Replacements for Fri, 17 Jan 20
 [11] arXiv:1807.02198 (replaced) [pdf, ps, other]

Title: The Radius of Metric SubregularityComments: 20 pagesJournalref: SetValued and Variational Analysis (2020)Subjects: Optimization and Control (math.OC)
 [12] arXiv:1808.03864 (replaced) [pdf, ps, other]

Title: Spectral norm of a symmetric tensor and its computationComments: 31 pages. arXiv admin note: substantial text overlap with arXiv:1608.01354, to appear in Mathematics of Computation, AMSSubjects: Optimization and Control (math.OC); Mathematical Physics (mathph)
 [13] arXiv:1809.09847 (replaced) [pdf, ps, other]

Title: SSPADE Done Right: Detailed Study of the Sparse Audio Declipper AlgorithmsSubjects: Optimization and Control (math.OC); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)
 [14] arXiv:1812.00734 (replaced) [pdf, other]

Title: Market Integration of HVDC Lines: Internalizing HVDC Losses in Market ClearingComments: Submitted to "IEEE Transactions on Power Systems" on December 3, 2018  Revised on May 28, 2019  Accepted on July 21, 2019  Published on January 7, 2020Journalref: IEEE Transactions on Power Systems, vol. 35, no. 1, pp. 451461, January 2020Subjects: Optimization and Control (math.OC)
 [15] arXiv:1904.01196 (replaced) [pdf, other]

Title: Linear Convergence of PrimalDual Gradient Methods and their Performance in Distributed OptimizationSubjects: Optimization and Control (math.OC)
 [16] arXiv:1904.07028 (replaced) [pdf, other]

Title: Euler's optimal profile problemSubjects: Optimization and Control (math.OC)
 [17] arXiv:1905.05257 (replaced) [pdf, other]

Title: OracleBased Algorithms for Binary TwoStage Robust OptimizationSubjects: Optimization and Control (math.OC)
 [18] arXiv:1905.05557 (replaced) [pdf, ps, other]

Title: An analytical bound on the fleet size in vehicle routing problems: a dynamic programming approachSubjects: Optimization and Control (math.OC); Discrete Mathematics (cs.DM)
 [19] arXiv:1907.02989 (replaced) [pdf, ps, other]

Title: An Optimality Gap Test for a Semidefinite Relaxation of a Quadratic Program with Two Quadratic ConstraintsSubjects: Optimization and Control (math.OC)
 [20] arXiv:1907.07580 (replaced) [pdf, ps, other]

Title: Featuredriven Improvement of Renewable Energy Forecasting and TradingComments: 10 pages, 6 figuresSubjects: Optimization and Control (math.OC); Applications (stat.AP); Machine Learning (stat.ML)
 [21] arXiv:1910.08123 (replaced) [pdf, other]

Title: Optimization and Learning with Information Streams: Timevarying Algorithms and ApplicationsComments: Accepted for publication in IEEE Signal Processing Magazine. Limit of 40 referencesSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Signal Processing (eess.SP)
 [22] arXiv:2001.04341 (replaced) [pdf, other]

Title: Information Newton's flow: secondorder optimization method in probability spaceComments: 43 pagesSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Machine Learning (stat.ML)
 [23] arXiv:2001.04729 (replaced) [pdf, ps, other]

Title: A unified method to decentralized state inference and fault diagnosis/prediction of discreteevent systemsAuthors: Kuize ZhangComments: 23 pages, 3 figuresSubjects: Optimization and Control (math.OC); Systems and Control (eess.SY)
 [24] arXiv:1901.00279 (replaced) [pdf, other]

Title: Elimination of All Bad Local Minima in Deep LearningComments: Accepted to appear in AISTATS 2020Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Optimization and Control (math.OC); Machine Learning (stat.ML)
 [25] arXiv:1902.04376 (replaced) [pdf, ps, other]

Title: An adaptive stochastic optimization algorithm for resource allocationComments: ALT2020, 45 pages, 9 figuresJournalref: Proceedings of Machine Learning Research (PMLR), volume 117, 2020Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC)
 [26] arXiv:1906.02351 (replaced) [pdf, other]

Title: On the Convergence of SARAH and BeyondSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
 [27] arXiv:1907.04371 (replaced) [pdf, other]

Title: Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk MinimizationComments: Accepted to appear in AISTATS 2020. Code available at: this https URLSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC)
 [28] arXiv:2001.04756 (replaced) [pdf, other]

Title: Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning ApproachSubjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Optimization and Control (math.OC); Machine Learning (stat.ML)
[ showing up to 2000 entries per page: fewer  more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, math, recent, 2001, contact, help (Access key information)