Abnormality-Driven Representation Learning for Radiology Imaging

Bibliographic Details
Title:	Abnormality-Driven Representation Learning for Radiology Imaging
Authors:	Ligero, Marta, Lenz, Tim, Wölflein, Georg, Nahhas, Omar S. M. El, Truhn, Daniel, Kather, Jakob Nikolas
Publication Year:	2024
Collection:	Computer Science
Subject Terms:	Computer Science - Computer Vision and Pattern Recognition
More Details:	To date, the most common approach for radiology deep learning pipelines is the use of end-to-end 3D networks based on models pre-trained on other tasks, followed by fine-tuning on the task at hand. In contrast, adjacent medical fields such as pathology, which focus on 2D images, have effectively adopted task-agnostic foundational models based on self-supervised learning (SSL), combined with weakly-supervised deep learning (DL). However, the field of radiology still lacks task-agnostic representation models due to the computational and data demands of 3D imaging and the anatomical complexity inherent to radiology scans. To address this gap, we propose CLEAR, a framework for radiology images that uses extracted embeddings from 2D slices along with attention-based aggregation for efficiently predicting clinical endpoints. As part of this framework, we introduce lesion-enhanced contrastive learning (LeCL), a novel approach to obtain visual representations driven by abnormalities in 2D axial slices across different locations of the CT scans. Specifically, we trained single-domain contrastive learning approaches using three different architectures: Vision Transformers, Vision State Space Models and Gated Convolutional Neural Networks. We evaluate our approach across three clinical tasks: tumor lesion location, lung disease detection, and patient staging, benchmarking against four state-of-the-art foundation models, including BiomedCLIP. Our findings demonstrate that CLEAR using representations learned through LeCL, outperforms existing foundation models, while being substantially more compute- and data-efficient.
Document Type:	Working Paper
Access URL:	http://arxiv.org/abs/2411.16803
Accession Number:	edsarx.2411.16803
Database:	arXiv

FullText	Text: Availability: 0 CustomLinks: – Url: http://arxiv.org/abs/2411.16803 Name: EDS - Arxiv Category: fullText Text: View this record from Arxiv MouseOverText: View this record from Arxiv – Url: https://resolver.ebsco.com/c/xy5jbn/result?sid=EBSCO:edsarx&genre=article&issn=&ISBN=&volume=&issue=&date=20241125&spage=&pages=&title=Abnormality-Driven Representation Learning for Radiology Imaging&atitle=Abnormality-Driven%20Representation%20Learning%20for%20Radiology%20Imaging&aulast=Ligero%2C%20Marta&id=DOI: Name: Full Text Finder (for New FTF UI) (s8985755) Category: fullText Text: Find It @ SCU Libraries MouseOverText: Find It @ SCU Libraries
Header	DbId: edsarx DbLabel: arXiv An: edsarx.2411.16803 RelevancyScore: 1128 AccessLevel: 3 PubType: Report PubTypeId: report PreciseRelevancyScore: 1128.033203125
IllustrationInfo
Items	– Name: Title Label: Title Group: Ti Data: Abnormality-Driven Representation Learning for Radiology Imaging – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Ligero%2C+Marta%22">Ligero, Marta</searchLink><br /><searchLink fieldCode="AR" term="%22Lenz%2C+Tim%22">Lenz, Tim</searchLink><br /><searchLink fieldCode="AR" term="%22Wölflein%2C+Georg%22">Wölflein, Georg</searchLink><br /><searchLink fieldCode="AR" term="%22Nahhas%2C+Omar+S%2E+M%2E+El%22">Nahhas, Omar S. M. El</searchLink><br /><searchLink fieldCode="AR" term="%22Truhn%2C+Daniel%22">Truhn, Daniel</searchLink><br /><searchLink fieldCode="AR" term="%22Kather%2C+Jakob+Nikolas%22">Kather, Jakob Nikolas</searchLink> – Name: DatePubCY Label: Publication Year Group: Date Data: 2024 – Name: Subset Label: Collection Group: HoldingsInfo Data: Computer Science – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22Computer+Science+-+Computer+Vision+and+Pattern+Recognition%22">Computer Science - Computer Vision and Pattern Recognition</searchLink> – Name: Abstract Label: Description Group: Ab Data: To date, the most common approach for radiology deep learning pipelines is the use of end-to-end 3D networks based on models pre-trained on other tasks, followed by fine-tuning on the task at hand. In contrast, adjacent medical fields such as pathology, which focus on 2D images, have effectively adopted task-agnostic foundational models based on self-supervised learning (SSL), combined with weakly-supervised deep learning (DL). However, the field of radiology still lacks task-agnostic representation models due to the computational and data demands of 3D imaging and the anatomical complexity inherent to radiology scans. To address this gap, we propose CLEAR, a framework for radiology images that uses extracted embeddings from 2D slices along with attention-based aggregation for efficiently predicting clinical endpoints. As part of this framework, we introduce lesion-enhanced contrastive learning (LeCL), a novel approach to obtain visual representations driven by abnormalities in 2D axial slices across different locations of the CT scans. Specifically, we trained single-domain contrastive learning approaches using three different architectures: Vision Transformers, Vision State Space Models and Gated Convolutional Neural Networks. We evaluate our approach across three clinical tasks: tumor lesion location, lung disease detection, and patient staging, benchmarking against four state-of-the-art foundation models, including BiomedCLIP. Our findings demonstrate that CLEAR using representations learned through LeCL, outperforms existing foundation models, while being substantially more compute- and data-efficient. – Name: TypeDocument Label: Document Type Group: TypDoc Data: Working Paper – Name: URL Label: Access URL Group: URL Data: <link linkTarget="URL" linkTerm="http://arxiv.org/abs/2411.16803" linkWindow="_blank">http://arxiv.org/abs/2411.16803</link> – Name: AN Label: Accession Number Group: ID Data: edsarx.2411.16803
PLink	https://login.libproxy.scu.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsarx&AN=edsarx.2411.16803
RecordInfo	BibRecord: BibEntity: Subjects: – SubjectFull: Computer Science - Computer Vision and Pattern Recognition Type: general Titles: – TitleFull: Abnormality-Driven Representation Learning for Radiology Imaging Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Ligero, Marta – PersonEntity: Name: NameFull: Lenz, Tim – PersonEntity: Name: NameFull: Wölflein, Georg – PersonEntity: Name: NameFull: Nahhas, Omar S. M. El – PersonEntity: Name: NameFull: Truhn, Daniel – PersonEntity: Name: NameFull: Kather, Jakob Nikolas IsPartOfRelationships: – BibEntity: Dates: – D: 25 M: 11 Type: published Y: 2024
ResultId	1