Modern AI Humbled By Vintage Atari Chess Game
Even with its remarkable progress, ChatGPT appears to have met its match in an unexpected opponent: an Atari game console nearly half a century old, at least when it comes to certain tasks.
Citrix software engineer Robert Caruso shared on LinkedIn how OpenAI's advanced chatbot was thoroughly defeated by an Atari 2600. The console was running Atari Chess, a game released back in 1979 during Jimmy Carter's presidency.
The Unlikely Showdown AI vs Retro Gaming
The Atari 2600, which launched in 1977 and was also known as the Atari Video Computer System, was instrumental in popularizing home gaming. This followed Atari's earlier "Pong" console. It's noteworthy that the Atari 2600 itself appeared 21 years after the MANIAC I supercomputer made history as the first machine to beat a human in a modified chess game. Given this timeline, one might expect today's sophisticated AI to easily outperform such vintage technology.
However, reality played out differently.
ChatGPTs Chess Catastrophe
Caruso stated, "ChatGPT got absolutely wrecked on the beginner level." He explained, "This was after a conversation we had regarding the history of AI in Chess which led to it volunteering to play Atari Chess. It wanted to find out how quickly it could beat a game that only thinks 1-2 moves ahead."
Despite being provided with a "baseline board" to understand the game and piece identification, ChatGPT struggled. It frequently confused rooks and bishops, misinterpreted moves, and consistently lost track of its piece positions. Caruso added that ChatGPT even blamed the Atari's icons, calling them "too abstract to recognize." However, switching to standard chess notation didn't improve its performance.
Caruso noted that for 90 minutes, ChatGPT committed blunders that "would get laughed out of a 3rd grade chess club." All the while, it repeatedly insisted it could win "if we just started over." The use of "we" by the chatbot, referring to itself and Caruso, adds a slightly unsettling layer to the interaction.
Atari Simplicity Versus AI Complexity
In stark contrast, "Atari’s humble 8-bit engine just did its thing," Caruso observed. "No language model. No flash. Just brute-force board evaluation and 1977 stubbornness."
Reflections on an AI Defeat
While there's no video footage of this epic takedown of ChatGPT by a vintage AI, the story itself provides a good chuckle about the current limitations of even advanced chatbots in specific gaming scenarios.
