LLMs Transforming Economic Research

The intersection of economics and artificial intelligence is rapidly evolving, with Large Language Models (LLMs) at the forefront of this transformation. This post delves into a collection of recent and influential academic papers that highlight how economists are harnessing LLMs for everything from complex data analysis to pedagogical innovation, while also navigating the inherent challenges these powerful tools present.

Advancing Economic Data Analysis with Deep Learning

A key paper anticipated for publication in 2025, "Deep Learning for Economists," explores the use of sophisticated AI for data analytics tasks that were previously too challenging or resource-intensive. Given that much of cutting-edge economics relies on uncovering new data insights, this development holds significant promise. However, it's also framed as a progression in computer-aided data mining rather than a complete revolution, acknowledging the field's existing computational tools.

LLMs as Simulators and Experimenters in Economics

Researchers are now investigating the potential of Large Language Models (LLMs) to generate novel data and test hypotheses, much like human experimenters. Two notable papers in this domain include:

• "Large language models as economic agents: what can we learn from homo silicus?"
• "Automated Social Science: Language Models as Scientist and Subjects" These studies explore how LLMs can simulate economic agents and behaviors, opening new avenues for research.

Exploring Generative AI Use Cases in Economic Research

The paper "Generative AI for Economic Research: Use Cases and Implications for Economists" by Korinek provides a comprehensive overview of how generative AI can be applied within the field. A significant update from December 2024, titled "LLMs Learn to Collaborate and Reason," supplements this work, highlighting advancements in LLM capabilities for collaboration and reasoning. This update was for the original article published in the Journal of Economic Literature 61 (4).

Early Guidance on LLMs in Economics Education

Recognized for its comprehensive and early insights, the paper "How to Learn and Teach Economics with Large Language Models, Including GPT" offered foundational perspectives on integrating LLMs into both learning and teaching economic principles.

Addressing AI Hallucinations in Economic Literature

A crucial study, "ChatGPT Hallucinates Non-existent Citations: Evidence from Economics," provided empirical evidence for the widely observed phenomenon of LLMs generating false or non-existent citations. This research validated concerns within the academic community. An important note indicates that while upcoming updates to web-enabled models are expected to reduce hallucination rates, the issue is not entirely resolved and remains an area for caution.

LLM Proficiency in Understanding Core Economic Concepts

A 2023 publication, "ChatGPT has Aced the Test of Understanding in College Economics: Now What?", revealed impressive capabilities of LLMs. The study found that ChatGPT performed exceptionally well, ranking in the 91st percentile for Microeconomics and the 99th percentile for Macroeconomics when benchmarked against students completing the Test of Understanding in College Economics (TUCE). This highlights the AI's strong grasp of fundamental economic principles.

Key Research Papers Discussed

The insights and discussions above are drawn from several important academic contributions. Here are the key papers mentioned:

• Deep Learning for Economists by M. Dell (2025). Journal of Economic Literature, 63(1), 5–58. Access Paper
• Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? by J. J. Horton (2023). arXiv Preprint arXiv:2301.07543. Access Paper
• Automated Social Science: Language Models as Scientist and Subjects by B. S. Manning, K. Zhu, & J. J. Horton (2024). National Bureau of Economic Research (Working Paper No. 32381). Access Paper
• Generative AI for Economic Research: Use Cases and Implications for Economists by A. Korinek (2023). Journal of Economic Literature, 61(4), 1281–1317. Access Paper
  • See also: LLMs Learn to Collaborate and Reason: December 2024 Update Access Update
• How to Learn and Teach Economics with Large Language Models, Including GPT by Tyler Cowen and Alexander T. Tabarrok (2023). GMU Working Paper in Economics No. 23-18. Access Paper on SSRN or via dx.doi.org
• ChatGPT Hallucinates Non-existent Citations: Evidence from Economics by J. Buchanan, S. Hill, & O. Shapoval (2023/2024). The American Economist, 69(1), 80-87. Access Paper
• ChatGPT has Aced the Test of Understanding in College Economics: Now What? by W. Geerling, G. D. Mateer, J. Wooten, & N. Damodaran (2023). The American Economist, 68(2), 233-245. Access Paper