extended · article · 2023

Assessing ChatGPT’s capacity for clinical decision support in pediatrics: A comparative study with pediatricians using KIDMAP of Rasch analysis

H. KAO; Tsair‐Wei Chien; Wen‐Chung Wang; Willy Chou; Julie Chi Chow; Mark Wilson

BACKGROUND: The application of large language models in clinical decision support (CDS) is an area that warrants further investigation. ChatGPT, a prominent large language models developed by OpenAI, has shown promising performance across various domains. However, there is limited research evaluating its use specifically in pediatric clinical decision-making. This study aimed to assess ChatGPT's potential as a CDS tool in pediatrics by evCDSaluating its performance on 8 common clinical symptom prompts. Study objectives were to answer the 2 research questions: the ChatGPT's overall grade in a range from A (high) to E (low) compared to a normal sample and the difference in assessment of ChatGPT between 2 pediatricians. METHODS: We compared ChatGPT's responses to 8 items related to clinical symptoms commonly encountered by pediatricians. Two pediatricians independently assessed the answers ...