References & Citations
Computer Science > Artificial Intelligence
Title: Self-Programming Artificial Intelligence Using Code-Generating Language Models
(Submitted on 30 Apr 2022 (v1), last revised 2 Feb 2023 (this version, v2))
Abstract: Recent progress in large-scale language models has enabled breakthroughs in previously intractable computer programming tasks. Prior work in meta-learning and neural architecture search has led to substantial successes across various task domains, spawning myriad approaches for algorithmically optimizing the design and learning dynamics of deep learning models. At the intersection of these research areas, we implement a code-generating language model with the ability to modify its own source code. Self-programming AI algorithms have been of interest since the dawn of AI itself. Although various theoretical formulations of generalized self-programming AI have been posed, no such system has been successfully implemented to date under real-world computational constraints. Applying AI-based code generation to AI itself, we develop and experimentally validate the first practical implementation of a self-programming AI system. We empirically show that a self-programming AI implemented using a code generation model can successfully modify its own source code to improve performance and program sub-models to perform auxiliary tasks. Our model can self-modify various properties including model architecture, computational capacity, and learning dynamics.
Submission history
From: Alex Sheng [view email][v1] Sat, 30 Apr 2022 05:44:34 GMT (76kb,D)
[v2] Thu, 2 Feb 2023 23:42:12 GMT (978kb,D)
Link back to: arXiv, form interface, contact.