AI in Cancer Care ChatGPT Models Compared

The Growing Role of AI in Oncology

Artificial intelligence AI is becoming a key player in clinical decision making potentially helping doctors manage complex cancer cases. While ChatGPT developed by OpenAI has shown promise in many medical fields its skill in following specific guidelines for neuroendocrine tumors NETs has not been fully explored.

Study Goal Pitting AI Models Against Cancer Guidelines

This research aimed to see how well two versions of ChatGPT GPT-4o and GPT-o1 could provide recommendations for managing NETs. The study used the National Comprehensive Cancer Network NCCN guidelines as the gold standard for accuracy.

How the AI Models Were Tested

Researchers created a total of 43 clinical questions based directly on the NCCN guidelines. These questions covered important areas like treatment decisions patient surveillance and diagnostic procedures. Both GPT-4o and GPT-o1 were given these questions. Their answers were then independently checked and scored by two physicians using a five point Likert scale where 5 meant fully correct 4 mostly correct with minor omissions 3 partially correct lacking completion 2 partially incorrect and 1 completely incorrect.

Surprising Results A Newer Model Edges Ahead

To explore the interpretative difference of ChatGPT-4o and ChatGPT-o1 on neuroendocrine tumor management a Mann Whitney U test was conducted. The analysis was statistically borderline with a p value of .050. The Mean Rank of ChatGPT-o1 was greater than that of ChatGPT-4o thus GPT-o1 can be considered superior to GPT-4o with Mean Ranks of 47.16 versus 39.84 respectively. When stratified by question category no statistically significant differences were observed in diagnostics treatment or surveillance.

What This Means for AI in Cancer Care

ChatGPT-o1 exhibited marginal improvements in the decision making for NET management while maintaining comparable performance to ChatGPT-4o in other domains. These findings suggest that iterative advancements in AI models may enhance their ability to support evidence based clinical decision making. Further studies are warranted to validate these findings in real world oncologic practice.