Google launches SIMA 2, the AI that plays video games

Last year, Google introduced SIMA (which stands for Scalable Instructable Multiworld Agent) to the world, a generalist AI capable of executing basic instructions within 3D virtual environments. 

The company recently showcased SIMA 2, an evolution of its predecessor, which has made significant strides thanks to the integration of Gemini models. Not only can it understand instructions via human language, but it can also decide on objectives, communicate with users, and improve its capabilities over time.

SIMA has evolved from performing about 600 simple operations, such as moving in a certain direction, using a ladder, or opening the map, by "watching" the screen and acting accordingly, to reasoning about what to do, analyzing the surrounding area more efficiently and realistically. The result is more akin to having a playmate than a trained dog to give orders to.

SIMA 2 can understand and execute complex tasks, but also respond to various forms of language, including emojis, and transfer knowledge learned in one context to apply it elsewhere. If interested, we refer you to the official Google DeepMind blog, with concrete examples of this new technology and more detailed explanations. Below is the presentation trailer.