But is it really all that useful?
Montezuma’s Revenge, the classic Atari platform game, has finally been fully solved by machine learning, researchers from Uber AI Labs claim.
Reinforcement learning (RL) eggheads have been fascinated with the old 1980s game for a while. It features an Indiana Jones-like character named Panama Joe, who goes around exploring tombs and fending off enemies to find hidden treasure. The game is an ideal environment for studying the problem of sparse rewards.
Montezuma’s Revenge is tricky as the rewards are spread out. A lot of intermediary steps are required, such as collecting various items to unlock rooms and defeating enemies, before a reward is given. It’s therefore not easy for machines to figure out: the path to success is not obvious.
But researchers at Uber – yes, that Uber – believe they have managed to do it with a new suite of algorithms dubbed Go-Explore. Unlike previous attempts to beat the game, this version of Panama Joe learns how to play the game without copying moves from human gameplay, a technique employed by DeepMind and OpenAI.
The Uber code also achieved the highest score so far, reaching over 400,000 points on average difficulty and managed to solve all three levels. Most researchers had trouble just trying to complete the first level of the game using their bots. The key thing here, therefore, is that Uber’s software player was able to solve all three levels, using new techniques, while previous research efforts by others don’t get much further than the first level.
