Bibliographic Details
| Title: |
Improved Automatic Diabetic Retinopathy Severity Classification Using Deep Multimodal Fusion of UWF-CFP and OCTA Images |
| Authors: |
Daho, Mostafa El Habib, Li, Yihao, Zeghlache, Rachid, Atse, Yapo Cedric, Boité, Hugo Le, Bonnin, Sophie, Cosette, Deborah, Deman, Pierre, Borderie, Laurent, Lepicard, Capucine, Tadayoni, Ramin, Cochener, Béatrice, Conze, Pierre-Henri, Lamard, Mathieu, Quellec, Gwenolé |
| Publication Year: |
2023 |
| Collection: |
Computer Science |
| Subject Terms: |
Electrical Engineering and Systems Science - Image and Video Processing, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning |
| More Details: |
Diabetic Retinopathy (DR), a prevalent and severe complication of diabetes, affects millions of individuals globally, underscoring the need for accurate and timely diagnosis. Recent advancements in imaging technologies, such as Ultra-WideField Color Fundus Photography (UWF-CFP) imaging and Optical Coherence Tomography Angiography (OCTA), provide opportunities for the early detection of DR but also pose significant challenges given the disparate nature of the data they produce. This study introduces a novel multimodal approach that leverages these imaging modalities to notably enhance DR classification. Our approach integrates 2D UWF-CFP images and 3D high-resolution 6x6 mm$^3$ OCTA (both structure and flow) images using a fusion of ResNet50 and 3D-ResNet50 models, with Squeeze-and-Excitation (SE) blocks to amplify relevant features. Additionally, to increase the model's generalization capabilities, a multimodal extension of Manifold Mixup, applied to concatenated multimodal features, is implemented. Experimental results demonstrate a remarkable enhancement in DR classification performance with the proposed multimodal approach compared to methods relying on a single modality only. The methodology laid out in this work holds substantial promise for facilitating more accurate, early detection of DR, potentially improving clinical outcomes for patients.
Comment: Accepted preprint for presentation at MICCAI-OMIA 20023, Vancouver, Canada |
| Document Type: |
Working Paper |
| DOI: |
10.1007/978-3-031-44013-7_2 |
| Access URL: |
http://arxiv.org/abs/2310.01912 |
| Accession Number: |
edsarx.2310.01912 |
| Database: |
arXiv |
