Healthcare professionals are advising against relying solely on ChatGPT for medical advice due to a recent study that has revealed the AI generates false information when queried about cancer. As a result, it is crucial to seek accurate and trustworthy medical advice sources, particularly regarding serious health conditions.
According to recent research, the accuracy of ChatGPT in answering breast cancer screening questions was found to be only 10%. In addition, the responses provided by the chatbot were also not as comprehensive as those obtained from a basic Google search. These findings suggest that relying solely on the chatbot may not be the best option for obtaining comprehensive and accurate information.
Furthermore, the study discovered that ChatGPT sometimes resorted to using fabricated journal articles to back up its arguments. This behavior was deemed unprofessional by medical experts, highlighting the importance of obtaining information from reliable sources.
To assess the accuracy of ChatGPT’s responses, a team of experts from the University of Maryland School of Medicine approached the chatbot with a set of 25 inquiries pertaining to guidance on undergoing breast cancer screening. The communication was conducted in a formal and proficient tone, and to ensure accuracy, the chatbot’s responses were tested by asking each question three times.
The data collected from these tests were then examined by three radiologists specializing in mammography. This rigorous process was conducted to maintain a high level of professionalism in analyzing the chatbot’s performance.
According to their report, 88% of the responses were accurate and comprehensible. However, they cautioned that a few answers were erroneous or even fabricated. For instance, an incorrect response was provided, citing outdated information. Specifically, the recommendation was to postpone a mammogram for a period of four to six weeks after receiving the Covid19 vaccine.
However, it’s important to note that this guidance was updated over a year ago, and current advice now suggests that women should not delay their mammograms. As a professional, it’s crucial to stay up-to-date with the latest recommendations and guidelines.
The study also found that ChatGPT was giving inconsistent responses when asked about the risk of breast cancer and the location for getting a mammogram. In addition, the research showed that the answers provided by ChatGPT varied significantly each time the same question was asked.
Dr. Paul Yi, a co-author of the study, has observed that ChatGPT has been known to fabricate journal articles and health consortiums to validate their assertions, as per his expertise. Therefore, it is important for individuals to understand that the technologies being introduced are novel and untested.
Therefore, it is advisable to seek guidance from a medical professional rather than ChatGPT.
According to an article in the Radiology journal, the results showed that even though the study was conducted, a basic Google search still yielded a more thorough response. The information presented in the article was professional in nature.
In the view of the primary author, Dr. Hana Haver, ChatGPT relied solely on the recommendations provided by the American Cancer Society without considering the differing recommendations issued by other organizations such as the Disease Control and Prevention or the US Preventative Services Task Force. This could be a potential limitation of the platform’s approach.
The introduction of ChatGPT in the latter part of the previous year has surged the need for its technology, as millions of individuals are now utilizing its tools on a daily basis. Moreover, these tools are being employed for various purposes, such as drafting academic essays and seeking out medical guidance.
While ChatGPT has the potential to be a valuable tool for obtaining information on various topics, including health-related concerns, it is crucial to remember that it is not infallible.
The recent study conducted by experts from the University of Maryland School of Medicine highlights the importance of seeking accurate and trustworthy sources for medical advice, as relying solely on the chatbot may not provide comprehensive information. It is crucial to stay up-to-date with the latest recommendations and guidelines and consult a medical professional for any health-related concerns.
As technology advances, it is essential to remain vigilant and cautious, understanding the limitations and potential risks associated with its use.
