Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Stochastic Multiple Target Sampling Gradient Descent
(Submitted on 4 Jun 2022 (v1), last revised 10 Feb 2023 (this version, v4))
Abstract: Sampling from an unnormalized target distribution is an essential problem with many applications in probabilistic inference. Stein Variational Gradient Descent (SVGD) has been shown to be a powerful method that iteratively updates a set of particles to approximate the distribution of interest. Furthermore, when analysing its asymptotic properties, SVGD reduces exactly to a single-objective optimization problem and can be viewed as a probabilistic version of this single-objective optimization problem. A natural question then arises: "Can we derive a probabilistic version of the multi-objective optimization?". To answer this question, we propose Stochastic Multiple Target Sampling Gradient Descent (MT-SGD), enabling us to sample from multiple unnormalized target distributions. Specifically, our MT-SGD conducts a flow of intermediate distributions gradually orienting to multiple target distributions, which allows the sampled particles to move to the joint high-likelihood region of the target distributions. Interestingly, the asymptotic analysis shows that our approach reduces exactly to the multiple-gradient descent algorithm for multi-objective optimization, as expected. Finally, we conduct comprehensive experiments to demonstrate the merit of our approach to multi-task learning.
Submission history
From: Phan Hoang [view email][v1] Sat, 4 Jun 2022 07:54:35 GMT (8564kb,D)
[v2] Sun, 12 Jun 2022 17:19:29 GMT (8592kb,D)
[v3] Fri, 23 Sep 2022 03:00:25 GMT (8612kb,D)
[v4] Fri, 10 Feb 2023 16:43:01 GMT (5642kb,D)
Link back to: arXiv, form interface, contact.