Federal Judges Blame AI For Error Ridden Court Orders

AI in the Dock: Judges Admit to Flawed Orders

In a significant disclosure, two federal judges have confirmed their staff's use of generative AI tools was responsible for error-filled court orders. The letters, made public by Senate Judiciary Chairman Chuck Grassley, shed light on the growing pains of integrating artificial intelligence into the legal system.

The judges, Henry T. Wingate of the Southern District of Mississippi and Julien Xavier Neals of the District of New Jersey, admitted that law clerks used AI to help draft orders. These documents were then mistakenly placed on the public docket before any senior review could take place. The resulting orders were plagued with serious mistakes, including misquoted text and references to individuals not involved in the cases, which led to speculation and official inquiries from Senator Grassley.

How It Happened: Unauthorized AI Use and Procedural Lapses

In their responses, both judges detailed how these critical errors occurred. Judge Neals confirmed that a temporary intern had used ChatGPT without authorization, an action that went against both chambers policy and the intern's law school rules. While his chambers had a verbal policy against using generative AI for legal research and drafting, it has since been formalized in writing.

Similarly, Judge Wingate explained that a clerk in his chambers used Perplexity as a "foundational drafting assistant to synthesize publicly available information." He was careful to note that no sensitive or non-public information was fed into the tool.

A common, critical failure in both instances was a breakdown of standard operating procedure. Judge Wingate stated, "the opinion that was docketed... was an early draft that had not gone through the standard review process." Both judges acknowledged that the drafts appeared on the public docket before undergoing the necessary and routine levels of review and cite-checking.

Preventing a Repeat: New Policies and Review Processes

To prevent future incidents, both judges have implemented stricter protocols. Judge Neals stated, "I now have a written unequivocal policy that applies to all law clerks and interns, pending definitive guidance... for appropriate AI usage."

Judge Wingate has instituted a more robust review system, creating a plan where "all draft opinions, orders, and memorandum decisions undergo a mandatory, independent review by a second law clerk before submission to me." Additionally, he now requires that all cited cases be printed directly from the legal database Westlaw and attached to final drafts for verification.

The Broader Impact: AI's Risky Role in the Legal World

The incidents highlight a growing challenge as AI tools become more accessible. A recent article in Cornell’s Journal of Empirical Legal Studies noted the "sharp rise in products incorporating artificial intelligence" within legal practice, finding that even specialized legal AI tools can overstate their capabilities.

Responding to the incident, a spokesperson for Perplexity stated that the tool "never claims to be 100% accurate" and that the company is "100% focused on building accurate AI." They added, "we strongly advise all AI users to check the work of their AI assistants. This is why we invented citations in AI in the first place." OpenAI, the creator of ChatGPT, did not immediately provide a comment.

Senator Grassley commended the judges for their honesty and expressed satisfaction with the new preventative measures. However, he issued a stern warning: "The judicial branch needs to develop more decisive, meaningful and permanent AI policies and guidelines. We can’t allow laziness, apathy or overreliance on artificial assistance to upend the Judiciary’s commitment to integrity and factual accuracy."