ChatGPT, an artificial intelligence tool, has limitations in providing medical diagnoses, treatments, or advice, but it can be a helpful resource for patients seeking medical information and guidance. Recent research has found that physicians could learn from the chatbot when it comes to patient communication.
In a study published on Friday, responses from ChatGPT were “preferred over physician responses and rated significantly higher for both quality and empathy” by a panel of licensed healthcare professionals who assessed responses to around 200 different medical questions. On average, ChatGPT scored 21% higher than physicians for the quality of responses and 41% more empathetic, according to the study.
Experts note that responses from ChatGPT were typically much longer than those from physicians, which could affect perceptions of quality and empathy. Additionally, the AI tool has the ability to customize responses to different literacy levels, making it advantageous for patients.
In comparing postoperative care instructions for eight standard pediatric procedures provided by ChatGPT, Google, and Stanford University, ChatGPT scored about even with Google in terms of understandability, both scoring better than 80%. However, while ChatGPT scored well in actionability (73%), Google responses were rated higher (83%).
Dr. David Asch, a professor of medicine and senior vice dean at the University of Pennsylvania, noted that ChatGPT is better viewed as a support for doctors than a guide for patients. Therefore, it is best used as “one step removed from the clinical encounter” in low-risk situations.
Asch cautioned that the AI tool still needs guardrails of truth to ensure that patients receive accurate and reliable information.
ChatGPT can provide valuable resources for patients seeking medical information and guidance. While it has limitations, the AI tool has advantages, including the ability to customize responses to different literacy levels. Still, it is better viewed as a support for doctors than a guide for patients.