AI Chatbot Excels at Colon Cancer Screening Questions

AI Enhances Patient Understanding of Cancer Screening

Study findings indicate that ChatGPT-4o offers accurate, complete, and trustworthy answers to questions about colorectal cancer (CRC) screening generated by patients.

Artificial intelligence (AI) can assist patients in better understanding cancer screening. According to a recent study, ChatGPT-4o consistently provided accurate and understandable responses to real questions from individuals eligible for colorectal cancer (CRC) screening. Through expert review and patient feedback on AI-generated answers, researchers discovered strong evidence supporting ChatGPT’s potential to encourage informed participation in CRC screening programs.

Study shows ChatGPT-4o provides accurate, complete, and trustworthy responses to patient-generated colorectal cancer (CRC) screening questions. | Image credit: irissca - stock.adobe.com

Study Confirms ChatGPTs Effectiveness

The cohort study detailing these findings is published in Endoscopy International Open.

The study's researchers noted, "In our study, despite a slight variability in responses, we found that ChatGPT-4o is still effective in providing accurate, complete, and understandable answers. Moreover, patients provided positive feedback about the completeness, comprehensibility, and trustworthiness of the responses, indicating their favorable perception of tool performance."

AI's Expanding Role in Early Cancer Detection

AI tools are showing promise in transforming how CRC is detected and understood. They offer potential ways to improve early diagnosis and patient engagement. For instance, research presented at the ASCO Gastrointestinal Cancers Symposium highlighted that the C the Signs model could identify individuals at high risk of CRC up to 5 years earlier than traditional methods by analyzing symptom patterns in electronic medical records. These developments showcase AI's growing ability not only to predict cancer risk but also to empower patients with accessible, timely, and trustworthy information, thereby encouraging earlier detection and intervention.

How Researchers Tested ChatGPTs Abilities

This particular study involved 10 individuals aged 50 to 69 years. These participants were eligible for the Italian national CRC screening program but were not currently taking part. Each person was given 4 standardized scenarios about common concerns related to CRC screening and was asked to create one question per scenario to get more information. These patient-generated questions were then put to ChatGPT-4o in two separate sessions to check for consistency and response quality. A panel of 5 senior experts in CRC screening independently assessed each AI-generated response using a 5-point Likert scale for accuracy, completeness, and comprehensibility.

Furthermore, the 10 participants who came up with the questions also reviewed ChatGPT-4o's responses. They rated them as complete, understandable, and trustworthy using a simple yes or no scale. All these evaluations were carried out independently, and the raters were not aware of each other's assessments.

Positive Feedback from Experts and Patients

The expert panel gave ChatGPT-4o's responses mean scores of 4.1 out of 5 for accuracy, 4.2 for completeness, and 4.3 for comprehensibility. These scores indicate a high-quality performance in key areas. Patient assessments were also very positive: 97.5% of responses were rated as complete, 95% as understandable, and 100% as trustworthy. Importantly, ChatGPT’s answers were consistent over time, with an 86.8% similarity between the two response sessions. This suggests reliable and reproducible performance when dealing with patient-generated CRC screening questions.

Important Considerations and Study Limitations

However, the researchers pointed out some limitations of the study. Firstly, they evaluated only one large language model (ChatGPT-4o) to ensure consistency with previous research, which restricted comparisons with other AI tools. Secondly, the small number of participants might have impacted the robustness of the findings. Lastly, since the study was conducted entirely in Italian, the results may not apply to other languages or cultural settings. Consequently, the researchers suggest that more extensive studies are necessary to validate these findings across different AI platforms and diverse populations.

AI as a Tool Not a Replacement for Doctors

Despite these limitations, the researchers believe the study indicates that ChatGPT performed well in answering CRC screening questions, even when patients used it directly.

The researchers emphasized, "Nevertheless, it is important to emphasize that this technology is not intended to replace professional medical advice. Most patients require face-to-face interactions with health care providers to discuss their concerns and receive necessary explanations. In addition, consulting a doctor is always needed to address complex issues involving health conditions and medication management and to provide personalized health care solutions."

References

1. Maida M, Mori Y, Fuccio L, et al. Exploring ChatGPT effectiveness in addressing direct patient queries on colorectal cancer screening. Endosc Int Open. 2025;13:a25689416. doi:10.1055/a-2568-9416
2. Steinzor P. Unlocking early colorectal cancer detection with artificial intelligence. AJMC. January 23, 2025. Accessed May 16, 2025. Link to article