Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations
(Submitted on 6 Feb 2023 (v1), last revised 24 May 2023 (this version, v2))
Abstract: Universality is a key hypothesis in mechanistic interpretability -- that different models learn similar features and circuits when trained on similar tasks. In this work, we study the universality hypothesis by examining how small neural networks learn to implement group composition. We present a novel algorithm by which neural networks may implement composition for any finite group via mathematical representation theory. We then show that networks consistently learn this algorithm by reverse engineering model logits and weights, and confirm our understanding using ablations. By studying networks of differing architectures trained on various groups, we find mixed evidence for universality: using our algorithm, we can completely characterize the family of circuits and features that networks learn on this task, but for a given network the precise circuits learned -- as well as the order they develop -- are arbitrary.
Submission history
From: Bilal Chughtai [view email][v1] Mon, 6 Feb 2023 18:59:20 GMT (1053kb,D)
[v2] Wed, 24 May 2023 22:13:13 GMT (1033kb,D)
Link back to: arXiv, form interface, contact.