How Penn State Researchers Are Supercharging AI for Science

While AI tools like ChatGPT can feel magical, the science behind them is a field of constant improvement. According to Rui Zhang, an assistant professor at Penn State's School of Electrical Engineering and Computer Science, there's always room to make these complex systems better. Zhang and his research group have recently authored three papers, set for publication at major AI conferences, that introduce groundbreaking approaches to AI optimization.

These papers, to be presented at the 63rd Annual Meeting of the Association for Computational Linguistics, the 2025 International Conference on Computer Vision, and the 13th International Conference on Learning Representations, focus on two key areas: automating AI prompts and enabling AI to understand high-resolution images.

Revolutionizing AI Interaction with Automated Prompts

Prompt engineering is the art of crafting specific inputs to get better outputs from AI. As Zhang explains, instead of a vague request like “summarize this article,” a more effective prompt is, “summarize this article in three bullet points for a high school student.” Being clear, specific, and goal-oriented is key.

However, manually creating the perfect prompt takes time and expertise. To solve this, Zhang's team developed GReaTer, a method that allows an AI system to automatically generate and refine its own prompts using advanced optimization algorithms. Building on this, they created GReaTerPrompt, a user-friendly, open-source toolkit that makes this powerful technology accessible to everyone.

Automating this process saves time and money, but more importantly, it boosts accuracy and allows AI to adapt to new tasks without human intervention. The team's evaluation showed that GReaTer significantly improved performance on complex reasoning and math tasks. In some tests, smaller language models optimized with GReaTer performed as well as much larger, more resource-intensive models. This has huge implications for tools like AI tutors, writing assistants, and customer support bots.

Rui Zhang (center) with research group members Ryo Kamoi and Yusen Zhang. Credit: Provided by Rui Zhang.

Enhancing AI Vision for High-Fidelity Science

Modern AI can describe images, but they often struggle with high-resolution visuals packed with crucial details. To address this, the researchers created HRScene, a new benchmark designed to test and improve how well AI models like GPT-4V and Gemini understand information-dense images.

High-resolution image understanding is vital for science. HRScene includes curated images from critical fields like radiology, plant phenotyping, remote sensing, and astronomy. By providing a standardized way to measure performance, this benchmark will accelerate the development of AI systems capable of analyzing fine-grained visual data with high accuracy.

Real-World Applications and Scientific Breakthroughs

The impact of this research spans numerous scientific and social domains. In healthcare, more accurate AI analysis of MRIs and CT scans could lead to earlier, more precise diagnoses. In agriculture, analyzing detailed plant images can help improve crop yields and promote sustainability. For environmental science and public safety, AI that can process high-resolution satellite imagery is a game-changer for disaster monitoring, urban planning, and climate research. Similarly, in astronomy, it can speed up the discovery of new celestial objects by analyzing vast telescope images.

By enabling AI to reliably process such detailed data, these advancements can accelerate scientific discovery, enhance public health, and improve our response to global challenges.

The Team Behind the Innovations

This work was a collaborative effort involving Penn State faculty and students, including assistant professor Wenpeng Yin, and doctoral and undergraduate students Yusen Zhang, Sarkar Snigdha Sarathi Das, and Wenliang Zheng, who led different facets of the research. The team also collaborated with researchers from Salesforce and received support from the U.S. National Science Foundation. To learn more about the importance of federal research funding, visit Research or Regress.