AI in Education Can ChatGPT Ace University Exams

AI Takes on University Exams A Groundbreaking Study

A recent study from George Mason University has put ChatGPT to the test, revealing that the AI performed nearly on par with real students on university-level health policy exams, albeit with some notable stumbles.

Sanja Avramovic, Department of Health Administration and Policy

Meet the Researchers Behind the AI Assessment

The research was spearheaded by Sanja Avramovic from the College of Public Health and Ivan Avramovic from the College of Engineering and Computing. Their work aimed to explore ChatGPT's potential as both a learning aid and a challenge to academic integrity in higher education.

"We wanted to understand what kinds of questions the tool can handle, and where it falls short," stated Sanja Avramovic, an associate professor of Health Administration and Policy. "That may help us design better teaching strategies and smarter assessments."

How ChatGPT Fared The Surprising Results

Researchers subjected ChatGPT to undergraduate, master’s, and PhD-level healthcare database exams. The AI was graded using the same rubrics applied to human students and impressively passed all three exam levels.

Strengths and Weaknesses AI in the Academic Arena

ChatGPT particularly excelled in SQL-based coding questions, even earning a remarkable 83.3% on a doctoral-level test. However, the AI tool faltered when questions required nuanced understanding or contextual reasoning, such as interpreting patient records across multiple visits.

National Recognition for Pioneering Research

The Avramovics’ insightful study earned them the 2024 Best Paper Award from the Journal of Health Administration Education. They are set to accept this honor this week at the annual meeting of the Association of University Programs in Health Administration (AUPHA).

The Future of AI in Learning A Supplement Not a Substitute

The findings suggest that while ChatGPT can be a valuable tool to supplement learning—by explaining code, checking logic, or generating alternative solutions—it cannot yet replicate the critical thinking or contextual awareness essential for students to master complex material.

Understanding vs Answering Key AI Limitations

"The current version of AI can answer many kinds of questions," the researchers conclude, "but that doesn’t mean it understands them." This distinction is crucial for educators and students navigating the evolving landscape of AI in education.

Dive Deeper Into The Research

Read the full study: Exploring the Potential Benefits and Limitations of Using an AI Text-Generation Tool in Education: An Examination of ChatGPT’s Performance on Assessments