Beyond the Hype The Real Limitations of ChatGPT

Mainstream media often portrays AI titans like ChatGPT as flawless game-changers, making it seem foolish to have any doubts about their capabilities. And to be fair, ChatGPT makes a fantastic first impression. It can provide thoughtful, well-organized, and seemingly accurate answers to detailed questions in mere seconds, leading many to incorporate it into their daily routines.

However, a recent and surprising experiment has highlighted some of its unexpected weaknesses.

An Unexpected Challenger The 1977 Atari

Earlier this year, software engineer Robert Caruso set up a fascinating chess match, as reported in the German news magazine Der Spiegel. The opponents: the cutting-edge ChatGPT versus a humble chess program running on a 1977 Atari 2600.

The vintage console was working with a mere 128 bytes of memory, allowing it to look only one or two moves ahead. In contrast, ChatGPT boasted that it understood chess perfectly and would be looking 10–15 moves ahead.

The match quickly turned into a rout, but not in the way one might expect. The Atari won with breathtaking ease, and its sophisticated opponent never stood a chance.

How a Modern AI Lost to Vintage Tech

ChatGPT made one serious blunder after another. It confused bishops for rooks and failed to recognize simple threats like a pawn fork. It struggled to follow the flow of the game, even with standard chess notation. Its mistakes were more egregious than those made by the elementary school students the author once coached.

So, how did a 21st-century Goliath get crushed by a 1970s David? It turns out that while ChatGPT is a master at drawing on vast digital resources and using probabilities to formulate text, it is not adept at following strict rules and engaging in logical thinking. Unlike specialized programs such as Deep Blue, which are designed for logic and can beat the best human players, ChatGPT’s strength lies in language, not strategy.

Deeper Flaws Beyond the Chessboard

A failure to play decent chess is just one reason why we need to view programs like ChatGPT as works in progress. There are broader, more significant concerns to consider.

First, the data these models train on comes from all corners of the internet, including its dubious and unsavory parts. This has led to some AI offerings, including ChatGPT, exhibiting racist tendencies and gender bias.

Additionally, as author Karen Hao notes in Empire of AI, a constant tension exists within OpenAI, the company behind ChatGPT. There is a conflict between the desire to build a safe system with ethical guardrails and the commercial drive to get products to market before competitors. It’s not guaranteed that the goal of serving humanity's best interests will always win.

Finally, there is the issue of AI's tendency to fabricate—or “hallucinate,” in industry terms. ChatGPT and similar programs sometimes provide complete falsehoods instead of actual facts.

A Call for Cautious Engagement

Given these circumstances, it’s best to remain cautious about the true merits and promise of this new technology. It needs to be used prudently, and its responses must be verified. Sharing valid and trustworthy information is a serious matter, and when it comes to AI, it’s a lot more than just a game.