Current browse context:
cs.CV
Change to browse by:
References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models
(Submitted on 11 Oct 2020 (v1), last revised 17 May 2021 (this version, v3))
Abstract: Contrastive learning is a form of self-supervision that can leverage unlabeled data to produce pretrained models. While contrastive learning has demonstrated promising results on natural image classification tasks, its application to medical imaging tasks like chest X-ray interpretation has been limited. In this work, we propose MoCo-CXR, which is an adaptation of the contrastive learning method Momentum Contrast (MoCo), to produce models with better representations and initializations for the detection of pathologies in chest X-rays. In detecting pleural effusion, we find that linear models trained on MoCo-CXR-pretrained representations outperform those without MoCo-CXR-pretrained representations, indicating that MoCo-CXR-pretrained representations are of higher-quality. End-to-end fine-tuning experiments reveal that a model initialized via MoCo-CXR-pretraining outperforms its non-MoCo-CXR-pretrained counterpart. We find that MoCo-CXR-pretraining provides the most benefit with limited labeled training data. Finally, we demonstrate similar results on a target Tuberculosis dataset unseen during pretraining, indicating that MoCo-CXR-pretraining endows models with representations and transferability that can be applied across chest X-ray datasets and tasks.
Submission history
From: Pranav Rajpurkar [view email][v1] Sun, 11 Oct 2020 21:42:10 GMT (2235kb,D)
[v2] Sun, 21 Feb 2021 02:54:06 GMT (2114kb,D)
[v3] Mon, 17 May 2021 20:39:42 GMT (4952kb,D)
Link back to: arXiv, form interface, contact.