A roadmap to fair and trustworthy prediction model validation in healthcare
| Title: | A roadmap to fair and trustworthy prediction model validation in healthcare |
|---|---|
| Authors: | Ning, Yilin, Volovici, Victor, Ong, Marcus Eng Hock, Goldstein, Benjamin Alan, Liu, Nan |
| Publication Year: | 2023 |
| Collection: | Computer Science |
| Subject Terms: | Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computers and Society |
| More Details: | A prediction model is most useful if it generalizes beyond the development data with external validations, but to what extent should it generalize remains unclear. In practice, prediction models are externally validated using data from very different settings, including populations from other health systems or countries, with predictably poor results. This may not be a fair reflection of the performance of the model which was designed for a specific target population or setting, and may be stretching the expected model generalizability. To address this, we suggest to externally validate a model using new data from the target population to ensure clear implications of validation performance on model reliability, whereas model generalizability to broader settings should be carefully investigated during model development instead of explored post-hoc. Based on this perspective, we propose a roadmap that facilitates the development and application of reliable, fair, and trustworthy artificial intelligence prediction models. Comment: 12 pages, 2 figures |
| Document Type: | Working Paper |
| Access URL: | http://arxiv.org/abs/2304.03779 |
| Accession Number: | edsarx.2304.03779 |
| Database: | arXiv |
| FullText | Text: Availability: 0 CustomLinks: – Url: http://arxiv.org/abs/2304.03779 Name: EDS - Arxiv Category: fullText Text: View this record from Arxiv MouseOverText: View this record from Arxiv – Url: https://resolver.ebsco.com/c/xy5jbn/result?sid=EBSCO:edsarx&genre=article&issn=&ISBN=&volume=&issue=&date=20230407&spage=&pages=&title=A roadmap to fair and trustworthy prediction model validation in healthcare&atitle=A%20roadmap%20to%20fair%20and%20trustworthy%20prediction%20model%20validation%20in%20healthcare&aulast=Ning%2C%20Yilin&id=DOI: Name: Full Text Finder (for New FTF UI) (s8985755) Category: fullText Text: Find It @ SCU Libraries MouseOverText: Find It @ SCU Libraries |
|---|---|
| Header | DbId: edsarx DbLabel: arXiv An: edsarx.2304.03779 RelevancyScore: 1051 AccessLevel: 3 PubType: Report PubTypeId: report PreciseRelevancyScore: 1051.01708984375 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: A roadmap to fair and trustworthy prediction model validation in healthcare – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Ning%2C+Yilin%22">Ning, Yilin</searchLink><br /><searchLink fieldCode="AR" term="%22Volovici%2C+Victor%22">Volovici, Victor</searchLink><br /><searchLink fieldCode="AR" term="%22Ong%2C+Marcus+Eng+Hock%22">Ong, Marcus Eng Hock</searchLink><br /><searchLink fieldCode="AR" term="%22Goldstein%2C+Benjamin+Alan%22">Goldstein, Benjamin Alan</searchLink><br /><searchLink fieldCode="AR" term="%22Liu%2C+Nan%22">Liu, Nan</searchLink> – Name: DatePubCY Label: Publication Year Group: Date Data: 2023 – Name: Subset Label: Collection Group: HoldingsInfo Data: Computer Science – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22Computer+Science+-+Machine+Learning%22">Computer Science - Machine Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Science+-+Artificial+Intelligence%22">Computer Science - Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Science+-+Computers+and+Society%22">Computer Science - Computers and Society</searchLink> – Name: Abstract Label: Description Group: Ab Data: A prediction model is most useful if it generalizes beyond the development data with external validations, but to what extent should it generalize remains unclear. In practice, prediction models are externally validated using data from very different settings, including populations from other health systems or countries, with predictably poor results. This may not be a fair reflection of the performance of the model which was designed for a specific target population or setting, and may be stretching the expected model generalizability. To address this, we suggest to externally validate a model using new data from the target population to ensure clear implications of validation performance on model reliability, whereas model generalizability to broader settings should be carefully investigated during model development instead of explored post-hoc. Based on this perspective, we propose a roadmap that facilitates the development and application of reliable, fair, and trustworthy artificial intelligence prediction models.<br />Comment: 12 pages, 2 figures – Name: TypeDocument Label: Document Type Group: TypDoc Data: Working Paper – Name: URL Label: Access URL Group: URL Data: <link linkTarget="URL" linkTerm="http://arxiv.org/abs/2304.03779" linkWindow="_blank">http://arxiv.org/abs/2304.03779</link> – Name: AN Label: Accession Number Group: ID Data: edsarx.2304.03779 |
| PLink | https://login.libproxy.scu.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsarx&AN=edsarx.2304.03779 |
| RecordInfo | BibRecord: BibEntity: Subjects: – SubjectFull: Computer Science - Machine Learning Type: general – SubjectFull: Computer Science - Artificial Intelligence Type: general – SubjectFull: Computer Science - Computers and Society Type: general Titles: – TitleFull: A roadmap to fair and trustworthy prediction model validation in healthcare Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Ning, Yilin – PersonEntity: Name: NameFull: Volovici, Victor – PersonEntity: Name: NameFull: Ong, Marcus Eng Hock – PersonEntity: Name: NameFull: Goldstein, Benjamin Alan – PersonEntity: Name: NameFull: Liu, Nan IsPartOfRelationships: – BibEntity: Dates: – D: 07 M: 04 Type: published Y: 2023 |
| ResultId | 1 |