AI Chats And Legal Privilege A Risky Combination

The AI Privilege Debate Begins

Earlier this month, Sam Altman, Co-founder and CEO of OpenAI, the company behind the widely-used ChatGPT platform, made a public call for what he termed "AI Privilege." This article explores the events leading to that call and examines whether, under current New Zealand law, chat logs with generative AI tools are subject to legal advice privilege. The conclusion, in general, is that they are not. This means that such chat logs may need to be disclosed to another party in litigation or to a regulator in response to a request.

As generative AI tools become increasingly prevalent for various commercial purposes, the issues outlined below highlight the critical need for businesses to establish clear policies and expectations regarding their use. In particular, considering potential quality concerns with "legal advice" provided by generative AI, businesses should ensure that all legal advice continues to be sought from and provided by an in-house or external human lawyer.

New York Times vs OpenAI A Catalyst for Change

In December 2023, The New York Times initiated a copyright infringement lawsuit against Microsoft, OpenAI, and its affiliated companies. The lawsuit alleges the unlawful use of millions of its articles to train the large language model that powers ChatGPT.

The claim details allegations that the defendants' unauthorized use of The Times's articles "constitutes free-riding on The Times's significant efforts and investment of human capital to gather this information" 1 and has resulted in economic harm through lost advertising and subscription revenue 2.

The legal proceedings aim to hold OpenAI and the other defendants accountable for billions of dollars in damages for the alleged unauthorized copying and use of The Times's "uniquely valuable works" 3. Furthermore, the plaintiffs are seeking an order for the destruction of any GPT or other large language models that incorporate The Times's copyrighted material 4.

OpenAI is defending against these claims, partly on the basis of the "fair use" doctrine, and has described the lawsuit as "baseless" 5.

A significant development in these proceedings occurred on 13 May 2025, when the Court issued an order mandating OpenAI to preserve and segregate all output log data that would typically be deleted. This order is effective immediately and remains in place until further notice from the Court 6.

Prior to this court order, OpenAI's policy stated that for users opting for a "temporary chat" or deleting their chat history, most data would be retained for 30 days before permanent erasure. However, following the court's directive, OpenAI is now required to retain certain users' conversations with ChatGPT indefinitely.

In response to this, Mr. Altman took to social media, advocating for the development of a comprehensive legal and ethical framework to protect privacy in communications with AI. He introduced the term "AI Privilege," arguing that interactions with AI should receive the same level of confidentiality as communications with a lawyer or a doctor.

AI Chats and Legal Privilege The New Zealand Perspective

This situation brings forth a crucial question: under New Zealand law, in what circumstances could parties be compelled to disclose legal questions posed by users to generative AI tools and the responses they received?

The answer begins with two fundamental concepts in New Zealand law:

• Under the High Court Rules, for instance, the High Court can order the discovery of "documents" that are or have been in a party's "control." Both terms are defined very broadly: "documents" encompass "information electronically recorded or stored, and information derived from that information," and "control" refers to possession of the document, a right to possess it, or a right to inspect or copy it. There are numerous precedents where the Court has ordered parties in litigation to disclose items like text messages or instant messaging chats stored on employees' devices 7. Regulators' powers for compulsory production are similarly extensive.
• Under the Evidence Act, legal advice privilege applies if a communication is:
  • made between a person and their legal adviser (which includes individuals enrolled and practicing as barristers and/or solicitors in New Zealand, and overseas practitioners);
  • intended to be confidential; and
  • made for the purpose of obtaining or providing professional legal services.

It should not be assumed that user chats with generative AI tools are automatically responsive to a discovery order or a regulator's request. However, if they are:

• they will fall within the definition of "documents" under the High Court Rules; and
• legal questions posed and responses received will generally not be protected by legal advice privilege. To state the obvious, even if other elements of legal advice privilege were met, ChatGPT does not fit the statutory definition of a "legal adviser."

Are There Loopholes Can AI Chats Ever Be Privileged

There are some limited circumstances in which generative AI chats might be subject to privilege. For example:

• At common law, legal advice privilege extends to preparatory materials created in anticipation of seeking legal advice, even if these materials are not actually shared with a lawyer. The 2006 codification of this law under the Evidence Act arguably narrowed the common law position: section 54, on its face, suggests the privilege applies only to communications actually made to a legal adviser (the Law Commission has recommended reinstating the common law position). Nevertheless, the common law position remains relevant outside the Evidence Act's application to a "proceeding." In such scenarios, if a generative AI chat was used to prepare a question for an actual legal adviser, it might be arguable that the chat is privileged.
• Preparatory materials privilege (also known as "litigation privilege") is broader than legal advice privilege and does not necessitate the involvement of a lawyer. If legal proceedings are underway or anticipated, a party can assert privilege under section 56 of the Evidence Act for, among other things, information compiled or prepared by the party for the dominant purpose of preparing for the proceeding. It is more conceivable that generative AI chats could meet this definition.

Navigating Legal Waters in the Age of AI

It is clear that these circumstances are narrow. Unless and until Mr. Altman's call for the recognition of an "AI Privilege" gains significant traction (which seems unlikely, at least in the New Zealand context), legal advice should continue to be sought from an actual human lawyer.

This underscores the importance for businesses using generative AI tools to establish clear policies and expectations regarding their use, especially when dealing with potentially sensitive or legally relevant information. Given the potential quality concerns associated with "legal advice" from AI, it remains crucial for businesses to ensure that all formal legal advice is obtained from qualified in-house or external legal professionals.