DeepMind Takes On Starcraft II, Transforming It Into An AI Research Environment

AI research is getting more and more advanced. We now have AI assisted apps that can predict user’s behavior by learning their day-to-day habits. You can get an answer before even asking a question. AI systems used in self-driving cars can make the car drive through real, live traffic, watching at the speed and acceleration, noticing nearby pedestrians and being able to stop the car before hitting one.

AI is used on mobile OSs in the form of virtual assistants. Mobile operating systems are a great ground for machine learning, since users spend every day with their mobile devices, enabling virtual assistants to learn about user’s daily behavior and make some decisions based on that knowledge. For instance, new machine learning system incorporated in the latest smartphone from Huawei, the Mate 9, can learn over time about the apps owner of the phone uses on a daily basis. Armed with that knowledge, the system can predict which apps will be used, even the time when they will be used; making the phone work faster by preparing the apps that will be opened in the near future.


For instance, if your daily morning routine includes reading your favorite news app, after which you want to see what’s happening on your favorite social network, the AI will prepare the social network app so that it can be launched instantly. Pretty interesting, but far away from what DeepMind is trying to do with Starcraft II.

If we look at them, video games are very complex systems. For learning the rules of a video game, you need lots of skills. You have to make many connections, to use both your short-term and long-term memory and to solve lots of problems, the foundation of intelligent behavior. DeepMind already did something similar, back when the Google-owned company constructed its AlphaGo program, that managed to beat a professional Go player for the first time ever. But Go is a simple game compared to the Starcraft II, especially in multiplayer.

In multiplayer, Starcraft II can go anywhere, depending on the behavior of your opponent. The game can go anywhere, there are numerous different units, three races, each one with unique mechanics, units and tactics, and lots of different maps, each one asking from the player to adapt in order to survive. DeepMind recognized that. In the post published on the official DeepMind site, the company stated that “DeepMind is on a scientific mission to push the boundaries of AI, developing programs that can learn to solve any complex problem without needing to be told how. Games are the perfect environment in which to do this, allowing us to develop and test smarter, more flexible AI algorithms quickly and efficiently, and also providing instant feedback on how we’re doing through scores. “


By playing video games, AI algorithms can evolve, learn through different scenarios, make decisions based on previous experience, but at the same time including the real-time info gotten from the game they currently play. One very important part of every AI is planning through time, and that’s where DeepMind saw the potential of Starcraft II. As the company states, “An agent that can play StarCraft will need to demonstrate effective use of memory, an ability to plan over a long time, and the capacity to adapt plans based on new information. Computers are capable of extremely fast control, but that doesn’t necessarily demonstrate intelligence, so agents must interact with the game within limits of human dexterity in terms of “Actions Per Minute”. StarCraft’s high-dimensional action space is quite different from those previously investigated in reinforcement learning research; to execute something as simple as “expand your base to some location”, one must coordinate mouse clicks, camera, and available resources.”

The AI playing Starcraft II don’t have to get any info about the game and its mechanics, it can learn over time, getting better and better by using its memory, as well as planning and various tactics learned from previous games against live opponents. This is quite different from the regular game bots because bots use the limited set of instructions (which contain game’s rules, and scripts for different situations) and play the game in the same way, over and over again.

The best thing about this initiative is that Starcraft II will be transformed into a proper AI research environment, available to everyone. “We’re particularly pleased that the environment we’ve worked with Blizzard to construct will be open and available to all researchers next year. We recognize the efforts of the developers and researchers from the Brood War community in recent years, and hope that this new, modern and flexible environment – supported directly by the team at Blizzard – will be widely used to advance the state-of-the-art.”

This will push AI research forward. The immense ways where the game can take you is phenomenal for AI researchers. They can test different algorithms, can put their AI “agents” in increasingly difficult scenarios, and can measure changes and advances. Since Starcraft II is a video game, Blizzard and DeepMind constructed a special API, allowing the AI to “see” the game, control units, and have access to every part of the game. Since the game has so many variables, engineers developed a “new image-based interface that outputs a simplified low-resolution RGB image data for map & minimap, and the option to break out features into separate “layers”, like terrain heightfield, unit type, unit health etc.”

AI researchers will need some time in order to develop an AI algorithm capable of defeating (or even being competitive) a human player, but since Starcraft II AI research environment will be offered to everyone, we might see some serious AI advances in the coming years. Now, the researchers must work on how to use the knowledge gained for playing a game in solving real-life problems.



Leave a Reply

Your email address will not be published. Required fields are marked *