We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Artificial Intelligence

Title: Arguments about Highly Reliable Agent Designs as a Useful Path to Artificial Intelligence Safety

Abstract: Several different approaches exist for ensuring the safety of future Transformative Artificial Intelligence (TAI) or Artificial Superintelligence (ASI) systems, and proponents of different approaches have made different and debated claims about the importance or usefulness of their work in the near term, and for future systems. Highly Reliable Agent Designs (HRAD) is one of the most controversial and ambitious approaches, championed by the Machine Intelligence Research Institute, among others, and various arguments have been made about whether and how it reduces risks from future AI systems. In order to reduce confusion in the debate about AI safety, here we build on a previous discussion by Rice which collects and presents four central arguments which are used to justify HRAD as a path towards safety of AI systems.
We have titled the arguments (1) incidental utility,(2) deconfusion, (3) precise specification, and (4) prediction. Each of these makes different, partly conflicting claims about how future AI systems can be risky. We have explained the assumptions and claims based on a review of published and informal literature, along with consultation with experts who have stated positions on the topic. Finally, we have briefly outlined arguments against each approach and against the agenda overall.
Comments: 14 pages and 2 figures + 6 pages for references and appendices
Subjects: Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
ACM classes: I.2.0; F.1.m
Cite as: arXiv:2201.02950 [cs.AI]
  (or arXiv:2201.02950v1 [cs.AI] for this version)

Submission history

From: David Manheim [view email]
[v1] Sun, 9 Jan 2022 07:42:37 GMT (22kb,D)

Link back to: arXiv, form interface, contact.