Summary
A recent experiment that pit ChatGPT against anAtari 2600in a chess match had a surprising conclusion, with ChatGPT getting “absolutely wrecked” by the old console. Despite theAtari2600 approaching its 50th anniversary, it seems that ChatGPT couldn’t figure out how to best it at the game. In fact, it hardly made it through the match at all.
ChatGPTand other AI LLMs have been touted as the future, with many users utilizing the services for everything from generating images, setting one’s schedule, or cheating on homework. However, there have also been a lot of significant issues with the technology as well, with some recent LLM models seeing an increase in “hallucinations,” or essentially the LLM producing information that’s simply wrong.
Now, an engineer named Robert Jr. Caruso has shared the results of his experiment pitting ChatGPT in achess match against an Atari 2600. According to Caruso, he was having aconversation with ChatGPT about chess, when the AI itself suggested taking on the Atari 2600 to demonstrate its own prowess in the game. However, the process was anything but smooth. Caruso says that ChatGPT repeatedly got confused about where the pieces were, which were under its control, and repeatedly made poor decisions, like sacrificing knights to pawns. ChatGPT reportedly complained that the Atari icons were “too abstract” for it to understand, but Caruso notes that even after switching to standard chess notation, ChatGPT still made the same blunders. In the end, after 90 minutes of struggling, ChatGPT actually forfeited the match.
Another AI Struggles To Play a Game
Some might argue that ChatGPT wasn’t intended for this particular task, and it’s not the only AI that’s had some difficulties trying to play a game recently. A ChatGPT user came up with an experiment to see if the Open AI o3 model could handle playingPokemon Red. While it has been making progress, it’s far slower in figuring out what to do compared to what a human player - and even many young kids. At the time of publication, theAI playingPokemon Redstill hadn’t reached Victory Road, and had spent a whopping 366 hours, or over 15 consecutive full days, trying to get there.
With that said, there are some AI that may fare against chess or games in general better than eitherOpenAI’s ChatGPT or o3 models. Google recently claimed that its own Google Gemini had beatPokemon Blue, which is certainly impressive. However, it also took 800 hours to complete its goal.
