Humanity has MIT researchers to thank for this defeat, chronicled in a paper they published in February, but Melee isn’t the only video game getting a lot of playtime from learning machines. AI software has cracked Super Mario Bros.; early Atari SA games such as Space Invaders; arcade mainstays Pac-Man and Mortal Kombat; even mobile favorite Angry Birds. Optimists say AI can help solve the world’s toughest problems, including cancer and climate change. So why are AI systems spending so much time gaming?
It’s all about data. Games allow AI software to tackle the kinds of complex logic problems found in the real world—uncertainty, negotiation, bluffing, cooperation—in carefully controlled environments, says Vlad Firoiu, who was part of the team that cracked Melee. Researchers can start their fledgling AI with relatively simple video game problems, run the tests thousands or millions of times, then gradually move on to more complex challenges as the system learns to handle the initial ones, according to Bloomberg.
“With games, you can generate as much data as you like,” says Demis Hassabis, chief executive officer of DeepMind Technologies Ltd., the London AI company owned by Google parent Alphabet Inc. “You want to hit the sweet spot, not too hard and not too easy for your current algorithms.” Over the past three years, DeepMind has trained software to master Atari games and beat the world’s best players at go, a strategic board game popular in Asia.
Game environments are ideal for what’s known as reinforcement learning, says John Schulman, a researcher at nonprofit lab OpenAI, which is backed by the likes of Tesla Inc.’s Elon Musk, Y Combinator co-founder Sam Altman, and venture capitalist Peter Thiel. Reinforcement learning means the machines figure out proper sequences of actions through processes of trial and error. Again, the controlled environment helps: It’s much safer to teach an algorithm how to drive with a racing game than with your car. An engineer at Otto, the driverless-truck company owned by Uber Technologies Inc., has tried teaching AI software to play Grand Theft Auto V, as has a team from Intel Corp.
Hassabis says DeepMind has focused on games because it believes true general AI will have to understand its presence in a physical landscape. The simulated environments in games are a way to do this without having to build robots, which can be a pain in the ass, he says: “They are slow. They break. You can’t run faster than real time. You can’t run millions of them in parallel.”
Some influential executives have turned their noses up at game research. “We are not pursuing AI to beat humans at games,” Microsoft Corp. CEO Satya Nadella said at his company’s developers conference in September. Instead, he said, Microsoft had staked much of its future on AI as a means to help solve “the most pressing problems of our society and economy.” Chris Bishop, who heads Microsoft’s AI research lab in Cambridge, England, says making competitive games the benchmark for AI also reinforces fears that smart computers threaten people.
That said, Microsoft has created Project Malmo, an AI research environment based on the video game Minecraft, which it happens to own. Bishop says Minecraft doesn’t feed the man-vs.-machine narrative because it has no set objectives and isn’t necessarily competitive.
Knocks on the gaming model haven’t deterred other AI researchers. Besides Grand Theft Auto V, they’re using dozens of games as tests, including StarCraft II, Montezuma’s Revenge, and Freeciv, a free game based on Sid Meier’s Civilization series. But, as OpenAI’s Schulman says, the real trick is developing AI that can solve not one game but any game you give it. Then the technology might be ready for the game of life, and not the Milton Bradley version.
Game Theories
Researchers are training computers to beat dozens of video games. Different games help hone different skills.
Super Smash Bros. Melee
Who MIT researchers
Skills Strategies for competing with multiple players and imperfect information
Freeciv
Who Arago, a German AI company
Skills Memory, long-term planning, and compensating for imperfect information in turn-based play
Project Malmo (Minecraft)
Who Microsoft
Skills Moving in 3D space from an embodied, first-person point of view, plus planning and cooperating with human players and other AI agents
Lab
Who DeepMind
Skills Navigating a mazelike 3D world from a first-person perspective, remembering routes, accounting for multiple allies or rivals
Grand Theft Auto V
Who Craig Quiter, a software engineer at Otto (the self-driving technology company Uber owns); Intel; Darmstadt University
Skills Driving—yes, driving, not murdering people
StarCraft: Brood War
Who Facebook and University of Oxford researchers
Skills Navigating a complicated visual landscape; also dealing with imperfect information, memory, and long-term planning in real-time play rather than making players wait their turn
StarCraft II
Who DeepMind and others
Skills Planning and priority-setting akin to that in the earlier StarCraft game, plus active competitions involving some of the world’s best pro gamers
Universe
Who OpenAI
Skills Visualizing the pixels on a screen and manipulating a virtual keyboard and mouse