“Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models”, 2022-12-21 (; backlinks):
[code] We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of 3 exams: Step 1, Step 2CK, and Step 3.
ChatGPT performed at or near the passing threshold for all 3 exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations.
These results suggest that large language models may have the potential to assist with medical education, and potentially, even clinical decision-making.