Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Self-supervised Learning from a Multi-view Perspective
(Submitted on 10 Jun 2020 (v1), last revised 22 Mar 2021 (this version, v4))
Abstract: As a subset of unsupervised representation learning, self-supervised representation learning adopts self-defined signals as supervision and uses the learned representation for downstream tasks, such as object detection and image captioning. Many proposed approaches for self-supervised learning follow naturally a multi-view perspective, where the input (e.g., original images) and the self-supervised signals (e.g., augmented images) can be seen as two redundant views of the data. Building from this multi-view perspective, this paper provides an information-theoretical framework to better understand the properties that encourage successful self-supervised learning. Specifically, we demonstrate that self-supervised learned representations can extract task-relevant information and discard task-irrelevant information. Our theoretical framework paves the way to a larger space of self-supervised learning objective design. In particular, we propose a composite objective that bridges the gap between prior contrastive and predictive learning objectives, and introduce an additional objective term to discard task-irrelevant information. To verify our analysis, we conduct controlled experiments to evaluate the impact of the composite objectives. We also explore our framework's empirical generalization beyond the multi-view perspective, where the cross-view redundancy may not be clearly observed.
Submission history
From: Yao-Hung Tsai [view email][v1] Wed, 10 Jun 2020 00:21:35 GMT (762kb,D)
[v2] Thu, 11 Jun 2020 01:54:15 GMT (761kb,D)
[v3] Sun, 4 Oct 2020 04:08:51 GMT (1408kb,D)
[v4] Mon, 22 Mar 2021 20:40:22 GMT (1410kb,D)
Link back to: arXiv, form interface, contact.