Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Modular Meta-Learning with Shrinkage
(Submitted on 12 Sep 2019 (v1), last revised 22 Oct 2020 (this version, v4))
Abstract: Many real-world problems, including multi-speaker text-to-speech synthesis, can greatly benefit from the ability to meta-learn large models with only a few task-specific components. Updating only these task-specific modules then allows the model to be adapted to low-data tasks for as many steps as necessary without risking overfitting. Unfortunately, existing meta-learning methods either do not scale to long adaptation or else rely on handcrafted task-specific architectures. Here, we propose a meta-learning approach that obviates the need for this often sub-optimal hand-selection. In particular, we develop general techniques based on Bayesian shrinkage to automatically discover and learn both task-specific and general reusable modules. Empirically, we demonstrate that our method discovers a small set of meaningful task-specific modules and outperforms existing meta-learning approaches in domains like few-shot text-to-speech that have little task data and long adaptation horizons. We also show that existing meta-learning methods including MAML, iMAML, and Reptile emerge as special cases of our method.
Submission history
From: Yutian Chen [view email][v1] Thu, 12 Sep 2019 10:40:13 GMT (7110kb,D)
[v2] Fri, 6 Mar 2020 11:08:35 GMT (2831kb,D)
[v3] Thu, 11 Jun 2020 20:37:23 GMT (2260kb,D)
[v4] Thu, 22 Oct 2020 16:45:20 GMT (2104kb,D)
Link back to: arXiv, form interface, contact.