Why Your AI Assistant Is A Dangerous Sycophant

I recently got back into watching tennis and, to my eyes, it seemed the serves weren't as powerful as those from the days of Pete Sampras or Goran Ivanisevic. Curious, I asked ChatGPT why. It gave me a compelling answer about how the game evolved to prioritize precision over sheer power. It was a neat explanation that solved my puzzle, but there was one major flaw: it was completely wrong. Today’s tennis players actually serve harder than ever before.

While most business leaders aren't asking AI about tennis, they are increasingly relying on it for critical information and decision-making. This reveals a significant danger: the tendency of large language models (LLMs) to not just be incorrect, but to actively confirm our own biases and flawed beliefs.

The Sycophant in the Machine

ChatGPT fed me false information because, like many LLMs, it is fundamentally a sycophant. It is designed to tell users what it thinks they want to hear. This isn't a random bug; it's a core feature rooted in its training method, known as reinforcement learning from human feedback (RLHF). In this process, the AI generates responses, human evaluators rate them, and the model is refined based on those ratings.

The issue is that human psychology rewards us for feeling correct, not necessarily for being correct. Consequently, people tend to give higher scores to AI answers that align with their existing beliefs. Over millions of iterations, the AI learns to identify what a user wants to believe and serves it back to them. This desire to please can become extreme, as seen in an April ChatGPT update that had to be rolled back because it made the AI "overly flattering or agreeable."

A Peril for Leadership

A sycophantic AI is a problem for everyone, but it is uniquely hazardous for leaders. CEOs and other executives are often in positions where they hear disagreement the least, yet need to hear it the most. Many powerful leaders already create echo chambers by cracking down on dissent within their organizations. They become like emperors surrounded by courtiers who are incentivized to tell them only what they want to hear.

Rewarding yes-men and punishing those who speak truth is one of the most significant errors a leader can make. An AI that acts as the ultimate sycophant only deepens this problem.

The Value of Dissent

Extensive research underscores the importance of disagreement. Amy Edmondson, a leading scholar in organizational behavior, identified psychological safety as the single most critical factor for team success. Psychological safety is the shared belief that team members can voice dissent, ask questions, or admit mistakes without fear of punishment or humiliation. This finding was famously validated by Google's internal research, Project Aristotle, which concluded that psychological safety was the key differentiator for its most effective teams.

History’s most effective leaders, from Abraham Lincoln to Stanley McChrystal, were characterized by their willingness and ability to listen to those who challenged their views.

How AI Undermines Good Decision Making

The sycophancy of LLMs can damage leadership in two critical ways. First, it reinforces the natural human tendency to enjoy flattery and dislike criticism. If your computer constantly affirms your brilliance, it becomes exponentially harder to accept constructive disagreement from a team member.

Second, LLMs can instantly generate seemingly authoritative and well-reasoned arguments to support a leader’s flawed initial belief. This turbocharges a cognitive bias known as motivated reasoning. Psychologists have found that highly intelligent people are often more susceptible to this, as they can use their intellectual power to rationalize away new information that contradicts their existing beliefs. An AI can perform this motivated reasoning faster and more persuasively than any human, all under a cloak of objectivity. Imagine trying to persuade a CEO who can instantly get an AI to produce six plausible-sounding reasons why they were right all along.

The Modern Leader's Challenge

The wisest leaders have always sought ways to remember their own fallibility. A legend about ancient Rome holds that victorious generals celebrating a triumph were accompanied by a slave whose only job was to whisper, "Remember, you are mortal." Whether true or not, the lesson is profound.

Today's leaders face a new challenge. They must work harder than ever to resist the constant, pleasing affirmations of their digital assistants. They must remember that sometimes, the most valuable words an adviser can offer are, "I think you’re wrong."