We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Multi-Modal Detection of Alzheimer's Disease from Speech and Text

Abstract: Reliable detection of the prodromal stages of Alzheimer's disease (AD) remains difficult even today because, unlike other neurocognitive impairments, there is no definitive diagnosis of AD in vivo. In this context, existing research has shown that patients often develop language impairment even in mild AD conditions. We propose a multimodal deep learning method that utilizes speech and the corresponding transcript simultaneously to detect AD. For audio signals, the proposed audio-based network, a convolutional neural network (CNN) based model, predicts the diagnosis for multiple speech segments, which are combined for the final prediction. Similarly, we use contextual embedding extracted from BERT concatenated with a CNN-generated embedding for classifying the transcript. The individual predictions of the two models are then combined to make the final classification. We also perform experiments to analyze the model performance when Automated Speech Recognition (ASR) system generated transcripts are used instead of manual transcription in the text-based model. The proposed method achieves 85.3% 10-fold cross-validation accuracy when trained and evaluated on the Dementiabank Pitt corpus.
Comments: 17 pages, 4 figures
Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL)
Cite as: arXiv:2012.00096 [cs.LG]
  (or arXiv:2012.00096v1 [cs.LG] for this version)

Submission history

From: Arnhav Datar [view email]
[v1] Mon, 30 Nov 2020 21:18:17 GMT (1220kb,D)
[v2] Thu, 17 Jun 2021 14:44:57 GMT (1955kb,D)
[v3] Sat, 31 Jul 2021 13:37:22 GMT (1969kb,D)

Link back to: arXiv, form interface, contact.