Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: From latent dynamics to meaningful representations
(Submitted on 2 Sep 2022 (v1), last revised 9 Apr 2024 (this version, v4))
Abstract: While representation learning has been central to the rise of machine learning and artificial intelligence, a key problem remains in making the learned representations meaningful. For this, the typical approach is to regularize the learned representation through prior probability distributions. However, such priors are usually unavailable or are ad hoc. To deal with this, recent efforts have shifted towards leveraging the insights from physical principles to guide the learning process. In this spirit, we propose a purely dynamics-constrained representation learning framework. Instead of relying on predefined probabilities, we restrict the latent representation to follow overdamped Langevin dynamics with a learnable transition density - a prior driven by statistical mechanics. We show this is a more natural constraint for representation learning in stochastic dynamical systems, with the crucial ability to uniquely identify the ground truth representation. We validate our framework for different systems including a real-world fluorescent DNA movie dataset. We show that our algorithm can uniquely identify orthogonal, isometric and meaningful latent representations.
Submission history
From: Dedi Wang [view email][v1] Fri, 2 Sep 2022 09:27:37 GMT (3160kb,AD)
[v2] Mon, 3 Apr 2023 16:15:58 GMT (9281kb,AD)
[v3] Mon, 11 Sep 2023 23:45:41 GMT (8936kb,AD)
[v4] Tue, 9 Apr 2024 18:34:52 GMT (9112kb,AD)
Link back to: arXiv, form interface, contact.