Bibliographic Details
| Title: |
ChatGPT as a prospective undergraduate and medical school student. |
| Authors: |
Giunti, Marco1 (AUTHOR) giunti@unica.it, Garavaglia, Fabrizia Giulia1 (AUTHOR), Giuntini, Roberto1 (AUTHOR), Sergioli, Giuseppe1 (AUTHOR), Pinna, Simone1 (AUTHOR) |
| Source: |
PLoS ONE. 10/23/2024, Vol. 19 Issue 10, p1-31. 31p. |
| Subject Terms: |
*MEDICAL school admission, *SAT (Educational test), *CHATGPT, *UNIVERSITY & college admission, *LOGICAL fallacies |
| Abstract: |
This article reports the results of an experiment conducted with ChatGPT to see how its performance compares to human performance on tests that require specific knowledge and skills, such as university admission tests. We chose a general undergraduate admission test and two tests for admission to biomedical programs: the Scholastic Assessment Test (SAT), the Cambridge BioMedical Admission Test (BMAT), and the Italian Medical School Admission Test (IMSAT). In particular, we looked closely at the difference in performance between ChatGPT-4 and its predecessor, ChatGPT-3.5, to assess its evolution. The performance of ChatGPT-4 showed a significant improvement over ChatGPT-3.5 and, compared to real students, was on average within the top 10% in the SAT test, while the score in the IMSAT test granted admission to the two highest ranked Italian medical schools. In addition to the performance analysis, we provide a qualitative analysis of incorrect answers and a classification of three different types of logical and computational errors made by ChatGPT-4, which reveal important weaknesses of the model. This provides insight into the skills needed to use these models effectively despite their weaknesses, and also suggests possible applications of our analysis in the field of education. [ABSTRACT FROM AUTHOR] |
|
Copyright of PLoS ONE is the property of Public Library of Science and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) |
| Database: |
Academic Search Complete |
|
Full text is not displayed to guests. |
Login for full access.
|
