Current browse context:
cs.HC
Change to browse by:
References & Citations
Computer Science > Human-Computer Interaction
Title: Human Perception of Audio Deepfakes
(Submitted on 20 Jul 2021 (v1), revised 30 Sep 2021 (this version, v2), latest version 6 Oct 2022 (v6))
Abstract: The recent emergence of deepfakes, computerized realistic multimedia fakes, brought the detection of manipulated and generated content to the forefront. While many machine learning models for deepfakes detection have been proposed, the human detection capabilities have remained far less explored. This is of special importance as human perception differs from machine perception and deepfakes are generally designed to fool the human. So far, this issue has only been addressed in the area of images and video.
To compare the ability of humans and machines in detecting audio deepfakes, we conducted an online gamified experiment in which we asked users to discern bonda-fide audio samples from spoofed audio, generated with a variety of algorithms. 200 users competed for 8976 game rounds with an artificial intelligence (AI) algorithm trained for audio deepfake detection. With the collected data we found that the machine generally outperforms the humans in detecting audio deepfakes, but that the converse holds for a certain attack type, for which humans are still more accurate. Furthermore, we found that younger participants are on average better at detecting audio deepfakes than older participants, while IT-professionals hold no advantage over laymen. We conclude that it is important to combine human and machine knowledge in order to improve audio deepfake detection.
Submission history
From: Nicolas Michael Müller [view email][v1] Tue, 20 Jul 2021 09:19:42 GMT (339kb,D)
[v2] Thu, 30 Sep 2021 13:15:52 GMT (321kb,D)
[v3] Mon, 28 Mar 2022 12:32:45 GMT (1471kb,D)
[v4] Mon, 1 Aug 2022 07:51:21 GMT (1552kb,D)
[v5] Wed, 5 Oct 2022 08:22:11 GMT (1605kb,D)
[v6] Thu, 6 Oct 2022 07:49:41 GMT (1605kb,D)
Link back to: arXiv, form interface, contact.