We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Scaling Scaling Laws with Board Games

Authors: Andy L. Jones
Abstract: The largest experiments in machine learning now require resources far beyond the budget of all but a few institutions. Fortunately, it has recently been shown that the results of these huge experiments can often be extrapolated from the results of a sequence of far smaller, cheaper experiments. In this work, we show that not only can the extrapolation be done based on the size of the model, but on the size of the problem as well. By conducting a sequence of experiments using AlphaZero and Hex, we show that the performance achievable with a fixed amount of compute degrades predictably as the game gets larger and harder. Along with our main result, we further show that the test-time and train-time compute available to an agent can be traded off while maintaining performance.
Subjects: Machine Learning (cs.LG); Multiagent Systems (cs.MA)
Cite as: arXiv:2104.03113 [cs.LG]
  (or arXiv:2104.03113v2 [cs.LG] for this version)

Submission history

From: Andy Jones [view email]
[v1] Wed, 7 Apr 2021 13:34:25 GMT (218kb,D)
[v2] Thu, 15 Apr 2021 10:03:37 GMT (229kb,D)

Link back to: arXiv, form interface, contact.