We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.AI

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Artificial Intelligence

Title: Navigating Human Language Models with Synthetic Agents

Abstract: Modern natural language models such as the GPT-2/GPT-3 contain tremendous amounts of information about human belief in a consistently testable form. If these models could be shown to accurately reflect the underlying beliefs of the human beings that produced the data used to train these models, then such models become a powerful sociological tool in ways that are distinct from traditional methods, such as interviews and surveys. In this study, We train a version of the GPT-2 on a corpora of historical chess games, and then "launch" clusters of synthetic agents into the model, using text strings to create context and orientation. We compare the trajectories contained in the text generated by the agents/model and compare that to the known ground truth of the chess board, move legality, and historical patterns of play. We find that the percentages of moves by piece using the model are substantially similar from human patterns. We further find that the model creates an accurate latent representation of the chessboard, and that it is possible to plot trajectories of legal moves across the board using this knowledge.
Comments: 8 pages, 6 figures, 2 tables, 1 algorithm
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multiagent Systems (cs.MA)
ACM classes: I.2; I.6; J.4
Cite as: arXiv:2008.04162 [cs.AI]
  (or arXiv:2008.04162v7 [cs.AI] for this version)

Submission history

From: Philip Feldman [view email]
[v1] Mon, 10 Aug 2020 14:39:53 GMT (2613kb,D)
[v2] Wed, 12 Aug 2020 15:09:36 GMT (3262kb,D)
[v3] Thu, 13 Aug 2020 13:42:00 GMT (2306kb,D)
[v4] Fri, 14 Aug 2020 11:12:43 GMT (2307kb,D)
[v5] Mon, 24 Aug 2020 19:18:21 GMT (2416kb,D)
[v6] Mon, 28 Sep 2020 15:40:41 GMT (2418kb,D)
[v7] Tue, 29 Sep 2020 09:57:33 GMT (2418kb,D)

Link back to: arXiv, form interface, contact.