References & Citations
Computer Science > Artificial Intelligence
Title: The Alignment Problem from a Deep Learning Perspective
(Submitted on 30 Aug 2022 (v1), last revised 19 Mar 2024 (this version, v6))
Abstract: In coming years or decades, artificial general intelligence (AGI) may surpass human capabilities at many critical tasks. We argue that, without substantial effort to prevent it, AGIs could learn to pursue goals that are in conflict (i.e. misaligned) with human interests. If trained like today's most capable models, AGIs could learn to act deceptively to receive higher reward, learn misaligned internally-represented goals which generalize beyond their fine-tuning distributions, and pursue those goals using power-seeking strategies. We review emerging evidence for these properties. AGIs with these properties would be difficult to align and may appear aligned even when they are not. Finally, we briefly outline how the deployment of misaligned AGIs might irreversibly undermine human control over the world, and we review research directions aimed at preventing this outcome.
Submission history
From: Sören Mindermann [view email][v1] Tue, 30 Aug 2022 02:12:47 GMT (48kb)
[v2] Wed, 14 Dec 2022 18:58:01 GMT (103kb)
[v3] Fri, 16 Dec 2022 18:50:31 GMT (74kb)
[v4] Wed, 22 Feb 2023 20:01:23 GMT (267kb)
[v5] Fri, 1 Sep 2023 20:09:09 GMT (279kb)
[v6] Tue, 19 Mar 2024 17:07:47 GMT (276kb)
Link back to: arXiv, form interface, contact.