AI Courtroom Drama Claude Fails Legal Test

Image Credits: Carol Yepes / Getty Images

Originally reported: 12:37 PM PDT · May 15, 2025

Anthropic Admits AI Error in Court Filing

A lawyer representing the AI company Anthropic has formally admitted to using an erroneous citation generated by Anthropic's own Claude AI chatbot. This admission was part of a court filing submitted on Thursday in a Northern California court, related to an ongoing legal battle with music publishers.

According to Anthropic's statement in the filing, which was first reported by Bloomberg, the Claude AI "hallucinated" the citation, creating it with "an inaccurate title and inaccurate authors." The company's legal team further explained that their "manual citation check" failed to catch this error, as well as several other inaccuracies originating from Claude's AI-driven hallucinations.

Anthropic has apologized for the mistake, characterizing it as "an honest citation mistake and not a fabrication of authority."

The Legal Showdown: AI vs. Copyright Holders

This development came to light earlier in the week when lawyers for Universal Music Group and other music publishers accused Anthropic's expert witness, company employee Olivia Chen, of using Claude to cite fabricated articles in her testimony. Following these allegations, Federal Judge Susan van Keulen ordered Anthropic to provide a response.

The lawsuit initiated by the music publishers is a significant case among several ongoing disputes between copyright owners and technology companies. These disputes center on the alleged misuse of copyrighted works for the purpose of training and developing generative AI tools.

A Pattern of AI Blunders in the Legal Field

This incident involving Anthropic's Claude is the latest in a series of cases where lawyers have encountered problems after using AI in legal proceedings. For instance, earlier this week, a California judge criticized two law firms for submitting "bogus AI-generated research" in his court. In a similar vein, an Australian lawyer was discovered in January to have used ChatGPT for preparing court documents, with the chatbot producing faulty and non-existent citations.

Despite Risks Investment in Legal AI Soars

Interestingly, these high-profile errors do not seem to be deterring investment in startups focused on automating legal work. Harvey, a company that employs generative AI models to assist lawyers, is reportedly in discussions to raise over $250 million. This potential funding round would place Harvey's valuation at an estimated $5 billion, signaling continued strong investor confidence in the AI legal tech sector.