The Hidden Energy Cost of Your AI Chatbot

With the explosion in popularity of tools like ChatGPT, which now fields over a billion prompts daily from nearly 200 million users, it's easy to think the answers appear from thin air. However, behind this digital magic is a massive and growing energy footprint.

Asking a large language model a question uses about 10 times the electricity needed for a regular Google search.

The Soaring Energy Demands of AI Data Centers

The artificial intelligence models that power these chatbots are housed in data centers, which are becoming significant energy consumers. In 2023, these centers accounted for a staggering 4.4% of all electricity use in the United States and about 1.5% of global energy consumption. As the demand for AI continues to surge, these figures are projected to at least double by 2030.

"Just three years ago, we didn't even have ChatGPT yet," notes Alex de Vries-Gao, a sustainability researcher and founder of Digiconomist. "And now we're talking about a technology that's going to be responsible for almost half of the electricity consumption by data centers globally."

Training vs Inference The Two Energy Guzzlers

So what makes these chatbots so power-hungry? According to Mosharaf Chowdhury, a computer scientist at the University of Michigan, the immense energy use stems from two key phases in an AI's life cycle: training and inference.

The Massive Scale of AI Training

Training is the initial phase where a large language model (LLM) is fed enormous datasets to learn patterns and make predictions. The prevailing philosophy is that bigger models with more data yield better results. "The models nowadays have gotten so large, they don't fit in a single GPU [graphics processing unit]; they don't fit in a single server," Chowdhury explains.

To illustrate the scale, a 2023 study found that training a model like OpenAI's GPT-4 required an estimated 50 gigawatt-hours of energy. That's enough electricity to power the entire city of San Francisco for three days. This process involves thousands of high-power GPUs running nonstop for weeks or even months.

The Relentless Power of Daily Inferences

The second phase, inference, is when the AI uses its training to respond to a user's prompt. While a single request uses less power than the training phase, the sheer volume is what drives up energy consumption. OpenAI has stated that ChatGPT users generate over 2.5 billion prompts every day. Add to this the usage from other widely used models, like Google's Gemini, which is set to become the default in Google Search, and the numbers become astronomical.

"So even in inference, you can't really save any energy," says Chowdhury. "The model is already massive, but we have a massive number of people using it."

The Call for Transparency and Accountability

Researchers are actively working to measure and reduce these energy demands. Chowdhury, for instance, maintains an ML Energy Leaderboard to track the energy consumption of various open-source models. However, a major hurdle is the lack of transparency from the tech giants behind the most popular platforms like Google, Microsoft, and Meta, who often keep their exact energy usage data private.

This secrecy makes it difficult to assess the true environmental impact. De Vries-Gao believes the solution lies with both consumers and regulators. Users can push for better transparency to make more energy-conscious choices, which in turn can drive policy changes that hold corporations accountable.

"One very fundamental problem with digital applications is that the impact is never transparent," de Vries-Gao states. "The ball is with policymakers to encourage disclosure so that the users can start doing something."