AI Coding Tools Might Make You Slower

The world of software engineering has been radically reshaped by AI in recent years. Tools like Copilot and Cursor have made writing code from scratch seem almost obsolete, with human developers now focusing on prompting AI models and refining the output. The prevailing belief is that this has massively accelerated development cycles. But does it?

The Surprising Slowdown: Does AI Hinder Expert Coders?

A new study published by METR challenges this conventional wisdom. The research aimed to measure how much AI assistance speeds up the work of experienced software developers, and the results were startling.

METR tracked 16 developers as they worked on complex software projects, both with and without AI tools. After completing their tasks, the developers estimated that AI had made them about 20% faster. However, the actual measurements told a different story: using AI had slowed them down by roughly 20%. The finding sent ripples of surprise through the AI community. "I was pretty skeptical that this study was worth running, because I thought that obviously we would see significant speedup," commented David Rein, a staffer at METR.

Why The Drop in Productivity?

The immediate technical reason seems to be that current Large Language Models (LLMs), while proficient at coding, often fail to grasp a developer's precise intent on the first try. This leads to a lengthy back-and-forth process of refinement that can take longer than simply writing the code manually.

Study participants also pointed to more human factors. Quentin Anthony, one of the developers in the experiment, described LLMs as a "big dopamine shortcut button." He explained the temptation: "Do you keep pressing the button that has a 1% chance of fixing everything? It's a lot more enjoyable than the grueling alternative." He also noted it's easy to get distracted by social media while waiting for the AI to generate its response.

What This Means for the Future of AI in Development

The study's authors caution against overgeneralizing these results. The research focused exclusively on experienced coders, and it's possible that novices may benefit more significantly from AI guidance. Furthermore, developers are still in the early stages of learning how to best leverage these powerful but quirky new tools.

Encouragingly, other METR research indicates that the complexity of software tasks AI can fully automate is doubling every seven months. This rapid improvement suggests that while today's AI might slow you down, the AI of tomorrow could be a very different story.

A Chip on His Shoulder: Nvidia's Stance on China Exports

In related AI news, Nvidia CEO Jensen Huang recently made headlines by stating on CNN that the U.S. government shouldn't worry about the Chinese military using his company's top-tier AI chips. "They simply can’t rely on it," he said, noting that access "could be, of course, limited at any time."

Huang has been a vocal critic of U.S. policies restricting the export of advanced graphics processing units (GPUs) to China. He argues these restrictions not only harm U.S. companies by cutting them off from a massive market but also push Beijing to accelerate the development of its own domestic chip industry. He has reportedly been lobbying against these policies, even meeting with former President Donald Trump.

However, research from Georgetown’s Center for Security and Emerging Technology, based on an analysis of 66,000 military purchasing records, confirms that the Chinese military does indeed use Nvidia chips. A significant black market has also emerged to smuggle the chips into China since the export controls were enacted.

AI in Action: Speeding Up Scientific Research

While AI may be slowing down some coders, it's accelerating work in other fields. At the AI company Anthropic, scientists are using their own AI assistant, Claude, to manage the deluge of academic literature.

Instead of reading countless papers, researchers now upload them to Claude and use the chatbot to summarize the key findings. "I've changed my habits of how I read papers," said Jan Leike, a senior researcher at Anthropic. He acknowledges that Claude can make mistakes, but points out, "if I just skim-read the paper, I'm also gonna get important stuff wrong sometimes." The real benefit, he says, is efficiency. "It allows me to read much more papers than I did before... you can just focus on the papers that actually matter."

Further Reading on AI's Future

For a deeper dive into the high-stakes world of AI partnerships, Steven Levy's piece in Wired, Microsoft and OpenAI's AGI Fight Is Bigger Than a Contract, is essential reading. It explores the critical "AGI clause" in the contract between the two giants, which could determine the future of their collaboration and provides fascinating insight into how they are attempting to define a technology that could change everything.