KPMG Built a Tax AI With a 100 Page Prompt

Image Credit: Shutterstock / Radware

When large language models like ChatGPT burst onto the scene in late 2022, leaders at the global consultancy firm KPMG saw both immense potential and significant risk. Their initial experiments quickly highlighted the dangers of using public AI tools in a corporate environment.

John Munnelly, KPMG's chief digital officer, recalled that early tests produced "really scary" results. One alarming discovery involved sensitive financial data residing on unsecured internal servers, a finding that immediately prompted the firm to halt its experiments, restrict access to public AI, and develop a more secure strategy.

Building a Private AI Platform

To move forward safely, KPMG set out to build a closed environment for its AI development. By securing software licenses with partners like OpenAI and Microsoft, the firm created a proprietary platform called KPMG Workbench. This system was designed for safety and flexibility, incorporating retrieval-augmented generation (RAG) and the ability to host multiple large language models (LLMs).

Instead of relying on a single AI provider, KPMG adopted a multi-vendor approach, spreading its usage across models from OpenAI, Google, Microsoft, Anthropic, and Meta. Throughout 2023, the company invested heavily in training its employees on effective prompt writing and interaction with AI systems.

The Birth of TaxBot: Powered by a 100-Page Prompt

By 2024, the Australian arm of KPMG was ready to build specialized AI agents. One of its most ambitious projects was TaxBot, a tool designed to streamline the preparation of complex tax advice.

The development process began by gathering years of partner-written advice that Munnelly described as being "stored all over the place," often scattered across individual laptops. This vast repository of expert knowledge was combined with Australia’s official tax code and fed into a RAG model to serve as the agent's brain.

Creating a reliable and accurate agent from this data was no simple task. According to Munnelly, the core of TaxBot is an astonishingly detailed 100-page prompt. This massive document, which took a dedicated team months to draft, acts as the master instruction set for the AI, guiding its analysis and document generation within the Workbench platform.

A Day's Work in Two Weeks

The result of this intensive effort is a system that can achieve remarkable efficiency. After receiving several inputs and consulting with human experts for guidance, TaxBot can generate a comprehensive 25-page client-ready document. Munnelly stated that the agent now accomplishes tasks in a single day that once took two weeks to complete—a productivity leap he called “very efficient.”

This speed is particularly valuable for clients involved in time-sensitive transactions like mergers and acquisitions. However, KPMG emphasizes the importance of human oversight. Only licensed tax agents are authorized to operate the tool, ensuring that its output is professionally reviewed and validated before it reaches a client.

Beyond pure efficiency, KPMG reports that the introduction of agents like TaxBot has boosted staff satisfaction by automating repetitive and time-consuming tasks. The project has also opened up an unexpected revenue stream, with some clients expressing interest in purchasing similar AI agents for their own operations. Despite these clear advantages, the firm admits that precisely measuring the full financial benefits of these AI initiatives remains a complex challenge.

As originally reported by The Register