The FIX Benchmark: Extracting Features Interpretable to eXperts
| Title: | The FIX Benchmark: Extracting Features Interpretable to eXperts |
|---|---|
| Authors: | Jin, Helen, Havaldar, Shreya, Kim, Chaehyeon, Xue, Anton, You, Weiqiu, Qu, Helen, Gatti, Marco, Hashimoto, Daniel A, Jain, Bhuvnesh, Madani, Amin, Sako, Masao, Ungar, Lyle, Wong, Eric |
| Publication Year: | 2024 |
| Collection: | Computer Science |
| Subject Terms: | Computer Science - Machine Learning, Computer Science - Artificial Intelligence |
| More Details: | Feature-based methods are commonly used to explain model predictions, but these methods often implicitly assume that interpretable features are readily available. However, this is often not the case for high-dimensional data, and it can be hard even for domain experts to mathematically specify which features are important. Can we instead automatically extract collections or groups of features that are aligned with expert knowledge? To address this gap, we present FIX (Features Interpretable to eXperts), a benchmark for measuring how well a collection of features aligns with expert knowledge. In collaboration with domain experts, we propose FIXScore, a unified expert alignment measure applicable to diverse real-world settings across cosmology, psychology, and medicine domains in vision, language, and time series data modalities. With FIXScore, we find that popular feature-based explanation methods have poor alignment with expert-specified knowledge, highlighting the need for new methods that can better identify features interpretable to experts. |
| Document Type: | Working Paper |
| Access URL: | http://arxiv.org/abs/2409.13684 |
| Accession Number: | edsarx.2409.13684 |
| Database: | arXiv |
| FullText | Text: Availability: 0 CustomLinks: – Url: http://arxiv.org/abs/2409.13684 Name: EDS - Arxiv Category: fullText Text: View this record from Arxiv MouseOverText: View this record from Arxiv – Url: https://resolver.ebsco.com/c/xy5jbn/result?sid=EBSCO:edsarx&genre=article&issn=&ISBN=&volume=&issue=&date=20240920&spage=&pages=&title=The FIX Benchmark: Extracting Features Interpretable to eXperts&atitle=The%20FIX%20Benchmark%3A%20Extracting%20Features%20Interpretable%20to%20eXperts&aulast=Jin%2C%20Helen&id=DOI: Name: Full Text Finder (for New FTF UI) (s8985755) Category: fullText Text: Find It @ SCU Libraries MouseOverText: Find It @ SCU Libraries |
|---|---|
| Header | DbId: edsarx DbLabel: arXiv An: edsarx.2409.13684 RelevancyScore: 1112 AccessLevel: 3 PubType: Report PubTypeId: report PreciseRelevancyScore: 1112.2578125 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: The FIX Benchmark: Extracting Features Interpretable to eXperts – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Jin%2C+Helen%22">Jin, Helen</searchLink><br /><searchLink fieldCode="AR" term="%22Havaldar%2C+Shreya%22">Havaldar, Shreya</searchLink><br /><searchLink fieldCode="AR" term="%22Kim%2C+Chaehyeon%22">Kim, Chaehyeon</searchLink><br /><searchLink fieldCode="AR" term="%22Xue%2C+Anton%22">Xue, Anton</searchLink><br /><searchLink fieldCode="AR" term="%22You%2C+Weiqiu%22">You, Weiqiu</searchLink><br /><searchLink fieldCode="AR" term="%22Qu%2C+Helen%22">Qu, Helen</searchLink><br /><searchLink fieldCode="AR" term="%22Gatti%2C+Marco%22">Gatti, Marco</searchLink><br /><searchLink fieldCode="AR" term="%22Hashimoto%2C+Daniel+A%22">Hashimoto, Daniel A</searchLink><br /><searchLink fieldCode="AR" term="%22Jain%2C+Bhuvnesh%22">Jain, Bhuvnesh</searchLink><br /><searchLink fieldCode="AR" term="%22Madani%2C+Amin%22">Madani, Amin</searchLink><br /><searchLink fieldCode="AR" term="%22Sako%2C+Masao%22">Sako, Masao</searchLink><br /><searchLink fieldCode="AR" term="%22Ungar%2C+Lyle%22">Ungar, Lyle</searchLink><br /><searchLink fieldCode="AR" term="%22Wong%2C+Eric%22">Wong, Eric</searchLink> – Name: DatePubCY Label: Publication Year Group: Date Data: 2024 – Name: Subset Label: Collection Group: HoldingsInfo Data: Computer Science – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22Computer+Science+-+Machine+Learning%22">Computer Science - Machine Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Science+-+Artificial+Intelligence%22">Computer Science - Artificial Intelligence</searchLink> – Name: Abstract Label: Description Group: Ab Data: Feature-based methods are commonly used to explain model predictions, but these methods often implicitly assume that interpretable features are readily available. However, this is often not the case for high-dimensional data, and it can be hard even for domain experts to mathematically specify which features are important. Can we instead automatically extract collections or groups of features that are aligned with expert knowledge? To address this gap, we present FIX (Features Interpretable to eXperts), a benchmark for measuring how well a collection of features aligns with expert knowledge. In collaboration with domain experts, we propose FIXScore, a unified expert alignment measure applicable to diverse real-world settings across cosmology, psychology, and medicine domains in vision, language, and time series data modalities. With FIXScore, we find that popular feature-based explanation methods have poor alignment with expert-specified knowledge, highlighting the need for new methods that can better identify features interpretable to experts. – Name: TypeDocument Label: Document Type Group: TypDoc Data: Working Paper – Name: URL Label: Access URL Group: URL Data: <link linkTarget="URL" linkTerm="http://arxiv.org/abs/2409.13684" linkWindow="_blank">http://arxiv.org/abs/2409.13684</link> – Name: AN Label: Accession Number Group: ID Data: edsarx.2409.13684 |
| PLink | https://login.libproxy.scu.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsarx&AN=edsarx.2409.13684 |
| RecordInfo | BibRecord: BibEntity: Subjects: – SubjectFull: Computer Science - Machine Learning Type: general – SubjectFull: Computer Science - Artificial Intelligence Type: general Titles: – TitleFull: The FIX Benchmark: Extracting Features Interpretable to eXperts Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Jin, Helen – PersonEntity: Name: NameFull: Havaldar, Shreya – PersonEntity: Name: NameFull: Kim, Chaehyeon – PersonEntity: Name: NameFull: Xue, Anton – PersonEntity: Name: NameFull: You, Weiqiu – PersonEntity: Name: NameFull: Qu, Helen – PersonEntity: Name: NameFull: Gatti, Marco – PersonEntity: Name: NameFull: Hashimoto, Daniel A – PersonEntity: Name: NameFull: Jain, Bhuvnesh – PersonEntity: Name: NameFull: Madani, Amin – PersonEntity: Name: NameFull: Sako, Masao – PersonEntity: Name: NameFull: Ungar, Lyle – PersonEntity: Name: NameFull: Wong, Eric IsPartOfRelationships: – BibEntity: Dates: – D: 20 M: 09 Type: published Y: 2024 |
| ResultId | 1 |