Current browse context:
cs.NE
Change to browse by:
References & Citations
Computer Science > Neural and Evolutionary Computing
Title: LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimization
(Submitted on 1 Jun 2023 (v1), last revised 12 Apr 2024 (this version, v8))
Abstract: Large Language Models (LLMs) have emerged as powerful tools capable of accomplishing a broad spectrum of tasks. Their abilities span numerous areas, and one area where they have made a significant impact is in the domain of code generation. Here, we propose using the coding abilities of LLMs to introduce meaningful variations to code defining neural networks. Meanwhile, Quality-Diversity (QD) algorithms are known to discover diverse and robust solutions. By merging the code-generating abilities of LLMs with the diversity and robustness of QD solutions, we introduce \texttt{LLMatic}, a Neural Architecture Search (NAS) algorithm. While LLMs struggle to conduct NAS directly through prompts, \texttt{LLMatic} uses a procedural approach, leveraging QD for prompts and network architecture to create diverse and high-performing networks. We test \texttt{LLMatic} on the CIFAR-10 and NAS-bench-201 benchmarks, demonstrating that it can produce competitive networks while evaluating just $2,000$ candidates, even without prior knowledge of the benchmark domain or exposure to any previous top-performing models for the benchmark. The open-sourced code is available in \url{this https URL}.
Submission history
From: Muhammad Umair Nasir Mr. [view email][v1] Thu, 1 Jun 2023 19:33:21 GMT (615kb,D)
[v2] Wed, 16 Aug 2023 15:49:48 GMT (4542kb,D)
[v3] Sat, 9 Sep 2023 18:58:26 GMT (4542kb,D)
[v4] Sun, 17 Sep 2023 15:31:15 GMT (4543kb,D)
[v5] Tue, 3 Oct 2023 07:43:30 GMT (914kb,D)
[v6] Wed, 4 Oct 2023 06:51:09 GMT (1305kb,D)
[v7] Wed, 10 Apr 2024 13:18:37 GMT (4356kb,D)
[v8] Fri, 12 Apr 2024 08:17:54 GMT (4356kb,D)
Link back to: arXiv, form interface, contact.