Current browse context:
eess.IV
Change to browse by:
References & Citations
Electrical Engineering and Systems Science > Image and Video Processing
Title: Alternating Cross-attention Vision-Language Model for Efficient Learning with Medical Image and Report without Curation
(Submitted on 10 Aug 2022 (this version), latest version 12 Apr 2023 (v4))
Abstract: Recent advances in vision-language pre-training have demonstrated astounding performances in diverse vision-language tasks, shedding a light on the long-standing problems of a comprehensive understanding of both visual and textual concepts in artificial intelligence research. However, there has been limited success in the application of vision-language pre-training in the medical domain, as the current vision-language models and learning strategies for photographic images and captions are not optimal to process the medical data which are usually insufficient in the amount and the diversity, which impedes successful learning of joint vision-language concepts. In this study, we introduce MAX-VL, a model tailored for efficient vision-language pre-training in the medical domain. We experimentally demonstrated that the pre-trained MAX-VL model outperforms the current state-of-the-art vision language models in various vision-language tasks. We also suggested the clinical utility for the diagnosis of newly emerging diseases and human error detection as well as showed the widespread applicability of the model in different domain data.
Submission history
From: Jong Chul Ye [view email][v1] Wed, 10 Aug 2022 04:35:58 GMT (12638kb,D)
[v2] Fri, 2 Sep 2022 01:03:58 GMT (23725kb,D)
[v3] Tue, 25 Oct 2022 13:27:22 GMT (8871kb)
[v4] Wed, 12 Apr 2023 10:58:04 GMT (19718kb,D)
Link back to: arXiv, form interface, contact.