Current browse context:
eess.IV
Change to browse by:
References & Citations
Electrical Engineering and Systems Science > Image and Video Processing
Title: Rethinking Annotation Granularity for Overcoming Shortcuts in Deep Learning-based Radiograph Diagnosis: A Multicenter Study
(Submitted on 21 Apr 2021 (v1), last revised 8 Nov 2022 (this version, v4))
Abstract: Two DL models were developed using radiograph-level annotations (yes or no disease) and fine-grained lesion-level annotations (lesion bounding boxes), respectively named CheXNet and CheXDet. The models' internal classification performance and lesion localization performance were compared on a testing set (n=2,922), external classification performance was compared on NIH-Google (n=4,376) and PadChest (n=24,536) datasets, and external lesion localization performance was compared on NIH-ChestX-ray14 dataset (n=880). The models were also compared to radiologists on a subset of the internal testing set (n=496). Given sufficient training data, both models performed comparably to radiologists. CheXDet achieved significant improvement for external classification, such as in classifying fracture on NIH-Google (CheXDet area under the ROC curve [AUC]: 0.67, CheXNet AUC: 0.51; p<.001) and PadChest (CheXDet AUC: 0.78, CheXNet AUC: 0.55; p<.001). CheXDet achieved higher lesion detection performance than CheXNet for most abnormalities on all datasets, such as in detecting pneumothorax on the internal set (CheXDet jacknife alternative free-response ROC-figure of merit [JAFROC-FOM]: 0.87, CheXNet JAFROC-FOM: 0.13; p<.001) and NIH-ChestX-ray14 (CheXDet JAFROC-FOM: 0.55, CheXNet JAFROC-FOM: 0.04; p<.001). To summarize, fine-grained annotations overcame shortcut learning and enabled DL models to identify correct lesion patterns, improving the models' generalizability.
Submission history
From: Luyang Luo [view email][v1] Wed, 21 Apr 2021 14:21:37 GMT (1644kb)
[v2] Thu, 8 Jul 2021 08:30:43 GMT (1198kb)
[v3] Thu, 4 Aug 2022 17:34:50 GMT (2059kb)
[v4] Tue, 8 Nov 2022 16:16:07 GMT (2059kb)
Link back to: arXiv, form interface, contact.