Assessing ChatGPT4 with and without retrieval-augmented generation in anticoagulation management for gastrointestinal procedures.
| Title: | Assessing ChatGPT4 with and without retrieval-augmented generation in anticoagulation management for gastrointestinal procedures. |
|---|---|
| Authors: | Malik, Sheza, Kharel, Himal, Dahiya, Dushyant S., Ali, Hassam, Blaney, Hanna, Singh, Achintya, Dhar, Jahnvi, Perisetti, Abhilash, Facciorusso, Antonio, Chandan, Saurabh, Mohan, Babu P. |
| Source: | Annals of Gastroenterology; 2024, Vol. 37 Issue 5, p514-526, 13p |
| Subject Terms: | CHATGPT, ARTIFICIAL intelligence, PHYSICIAN-patient relations, ENDOSCOPIC surgery, GASTROENTEROLOGISTS, ENDOSCOPIC retrograde cholangiopancreatography |
| Abstract: | Background In view of the growing complexity of managing anticoagulation for patients undergoing gastrointestinal (GI) procedures, this study evaluated ChatGPT-4's ability to provide accurate medical guidance, comparing it with its prior artificial intelligence (AI) models (ChatGPT-3.5) and the retrieval-augmented generation (RAG)-supported model (ChatGPT4-RAG). Method Thirty-six anticoagulation-related questions, based on professional guidelines, were answered by ChatGPT-4. Nine gastroenterologists assessed these responses for accuracy and relevance. ChatGPT-4's performance was also compared to that of ChatGPT-3.5 and ChatGPT4-RAG. Additionally, a survey was conducted to understand gastroenterologists' perceptions of ChatGPT-4. Results ChatGPT-4's responses showed significantly better accuracy and coherence compared to ChatGPT-3.5, with 30.5% of responses fully accurate and 47.2% generally accurate. ChatGPT4- RAG demonstrated a higher ability to integrate current information, achieving 75% full accuracy. Notably, for diagnostic and therapeutic esophagogastroduodenoscopy, 51.8% of responses were fully accurate; for endoscopic retrograde cholangiopancreatography with and without stent placement, 42.8% were fully accurate; and for diagnostic and therapeutic colonoscopy, 50% were fully accurate. Conclusions ChatGPT4-RAG significantly advances anticoagulation management in endoscopic procedures, offering reliable and precise medical guidance. However, medicolegal considerations mean that a 75% full accuracy rate remains inadequate for independent clinical decision-making. AI may be more appropriately utilized to support and confirm clinicians' decisions, rather than replace them. Further evaluation is essential to maintain patient confidentiality and the integrity of the physician-patient relationship. [ABSTRACT FROM AUTHOR] |
| Copyright of Annals of Gastroenterology is the property of Hellenic Society of Gastroenterology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Complementary Index |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://resolver.ebsco.com/c/xy5jbn/result?sid=EBSCO:edb&genre=article&issn=11087471&ISBN=&volume=37&issue=5&date=20240901&spage=514&pages=514-526&title=Annals of Gastroenterology&atitle=Assessing%20ChatGPT4%20with%20and%20without%20retrieval-augmented%20generation%20in%20anticoagulation%20management%20for%20gastrointestinal%20procedures.&aulast=Malik%2C%20Sheza&id=DOI:10.20524/aog.2024.0907 Name: Full Text Finder (for New FTF UI) (s8985755) Category: fullText Text: Find It @ SCU Libraries MouseOverText: Find It @ SCU Libraries |
|---|---|
| Header | DbId: edb DbLabel: Complementary Index An: 181013072 RelevancyScore: 1041 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 1040.51928710938 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Assessing ChatGPT4 with and without retrieval-augmented generation in anticoagulation management for gastrointestinal procedures. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Malik%2C+Sheza%22">Malik, Sheza</searchLink><br /><searchLink fieldCode="AR" term="%22Kharel%2C+Himal%22">Kharel, Himal</searchLink><br /><searchLink fieldCode="AR" term="%22Dahiya%2C+Dushyant+S%2E%22">Dahiya, Dushyant S.</searchLink><br /><searchLink fieldCode="AR" term="%22Ali%2C+Hassam%22">Ali, Hassam</searchLink><br /><searchLink fieldCode="AR" term="%22Blaney%2C+Hanna%22">Blaney, Hanna</searchLink><br /><searchLink fieldCode="AR" term="%22Singh%2C+Achintya%22">Singh, Achintya</searchLink><br /><searchLink fieldCode="AR" term="%22Dhar%2C+Jahnvi%22">Dhar, Jahnvi</searchLink><br /><searchLink fieldCode="AR" term="%22Perisetti%2C+Abhilash%22">Perisetti, Abhilash</searchLink><br /><searchLink fieldCode="AR" term="%22Facciorusso%2C+Antonio%22">Facciorusso, Antonio</searchLink><br /><searchLink fieldCode="AR" term="%22Chandan%2C+Saurabh%22">Chandan, Saurabh</searchLink><br /><searchLink fieldCode="AR" term="%22Mohan%2C+Babu+P%2E%22">Mohan, Babu P.</searchLink> – Name: TitleSource Label: Source Group: Src Data: Annals of Gastroenterology; 2024, Vol. 37 Issue 5, p514-526, 13p – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22CHATGPT%22">CHATGPT</searchLink><br /><searchLink fieldCode="DE" term="%22ARTIFICIAL+intelligence%22">ARTIFICIAL intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22PHYSICIAN-patient+relations%22">PHYSICIAN-patient relations</searchLink><br /><searchLink fieldCode="DE" term="%22ENDOSCOPIC+surgery%22">ENDOSCOPIC surgery</searchLink><br /><searchLink fieldCode="DE" term="%22GASTROENTEROLOGISTS%22">GASTROENTEROLOGISTS</searchLink><br /><searchLink fieldCode="DE" term="%22ENDOSCOPIC+retrograde+cholangiopancreatography%22">ENDOSCOPIC retrograde cholangiopancreatography</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Background In view of the growing complexity of managing anticoagulation for patients undergoing gastrointestinal (GI) procedures, this study evaluated ChatGPT-4's ability to provide accurate medical guidance, comparing it with its prior artificial intelligence (AI) models (ChatGPT-3.5) and the retrieval-augmented generation (RAG)-supported model (ChatGPT4-RAG). Method Thirty-six anticoagulation-related questions, based on professional guidelines, were answered by ChatGPT-4. Nine gastroenterologists assessed these responses for accuracy and relevance. ChatGPT-4's performance was also compared to that of ChatGPT-3.5 and ChatGPT4-RAG. Additionally, a survey was conducted to understand gastroenterologists' perceptions of ChatGPT-4. Results ChatGPT-4's responses showed significantly better accuracy and coherence compared to ChatGPT-3.5, with 30.5% of responses fully accurate and 47.2% generally accurate. ChatGPT4- RAG demonstrated a higher ability to integrate current information, achieving 75% full accuracy. Notably, for diagnostic and therapeutic esophagogastroduodenoscopy, 51.8% of responses were fully accurate; for endoscopic retrograde cholangiopancreatography with and without stent placement, 42.8% were fully accurate; and for diagnostic and therapeutic colonoscopy, 50% were fully accurate. Conclusions ChatGPT4-RAG significantly advances anticoagulation management in endoscopic procedures, offering reliable and precise medical guidance. However, medicolegal considerations mean that a 75% full accuracy rate remains inadequate for independent clinical decision-making. AI may be more appropriately utilized to support and confirm clinicians' decisions, rather than replace them. Further evaluation is essential to maintain patient confidentiality and the integrity of the physician-patient relationship. [ABSTRACT FROM AUTHOR] – Name: Abstract Label: Group: Ab Data: <i>Copyright of Annals of Gastroenterology is the property of Hellenic Society of Gastroenterology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://login.libproxy.scu.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edb&AN=181013072 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.20524/aog.2024.0907 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 13 StartPage: 514 Subjects: – SubjectFull: CHATGPT Type: general – SubjectFull: ARTIFICIAL intelligence Type: general – SubjectFull: PHYSICIAN-patient relations Type: general – SubjectFull: ENDOSCOPIC surgery Type: general – SubjectFull: GASTROENTEROLOGISTS Type: general – SubjectFull: ENDOSCOPIC retrograde cholangiopancreatography Type: general Titles: – TitleFull: Assessing ChatGPT4 with and without retrieval-augmented generation in anticoagulation management for gastrointestinal procedures. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Malik, Sheza – PersonEntity: Name: NameFull: Kharel, Himal – PersonEntity: Name: NameFull: Dahiya, Dushyant S. – PersonEntity: Name: NameFull: Ali, Hassam – PersonEntity: Name: NameFull: Blaney, Hanna – PersonEntity: Name: NameFull: Singh, Achintya – PersonEntity: Name: NameFull: Dhar, Jahnvi – PersonEntity: Name: NameFull: Perisetti, Abhilash – PersonEntity: Name: NameFull: Facciorusso, Antonio – PersonEntity: Name: NameFull: Chandan, Saurabh – PersonEntity: Name: NameFull: Mohan, Babu P. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 09 Text: 2024 Type: published Y: 2024 Identifiers: – Type: issn-print Value: 11087471 Numbering: – Type: volume Value: 37 – Type: issue Value: 5 Titles: – TitleFull: Annals of Gastroenterology Type: main |
| ResultId | 1 |