Google introduces the Gemini 2.0 AI model designed for agentic experiences.

On Wednesday, Google unveiled Gemini 2.0, its most advanced AI model to date, designed to support the emerging “agentic era.” Featuring capabilities such as native image and audio output, multimodal support, and advanced tool integration, Gemini 2.0 is aimed at creating more powerful AI agents, bringing Google closer to its vision of a universal assistant.

Gemini 2.0 Flash: Enhanced Performance and New Features

Gemini 2.0 Flash improves upon its predecessor, 1.5 Flash, offering significantly faster performance—twice as fast as the 1.5 Pro model. It supports multimodal inputs like images, videos, and audio, and outputs such as generated images and multilingual text-to-speech (TTS) audio. Additionally, it allows for the native use of tools, including Google Search and third-party functions, further boosting its versatility.

Project Astra: This prototype, designed for Android devices, features enhanced multilingual dialogue, tool integration (such as Google Search, Lens, and Maps), and up to 10 minutes of in-session memory. It also reduces latency for more natural conversations. Google plans to conduct further testing on prototype glasses.

Project Mariner: Focused on improving human-agent interaction for web tasks, this project uses Gemini 2.0 to process web elements such as text, images, and code. Although still in early development, it has already achieved an 83.5% success rate in web task execution. The project prioritizes enhancing efficiency and safety, incorporating user confirmation for sensitive actions.

Jules for Developers: This experimental AI agent, integrated into GitHub, helps developers by identifying issues, suggesting solutions, and executing tasks under supervision. As part of Google’s broader efforts to integrate AI across various fields, Jules aims to assist in software development.

AI Agents in Games and Beyond: Drawing on DeepMind’s gaming expertise, Google is developing AI agents capable of reasoning based on game actions and offering real-time suggestions. These agents are being tested in games like Clash of Clans and Hay Day, in partnership with developers such as Supercell. Google is also exploring Gemini 2.0’s applications for robotics and spatial reasoning in physical environments.

Responsible AI Development with Gemini 2.0: Google is committed to responsible AI development, emphasizing safety and security as it explores new agentic capabilities. Key measures include:

• Collaboration with the Responsibility and Safety Committee (RSC): Google’s internal review group evaluates potential risks and safety protocols.
• AI-assisted Red Teaming: The advanced reasoning capabilities of Gemini 2.0 allow for automated risk assessments, enhancing safety at scale.
• Multimodal Safety: Training Gemini 2.0 to handle a variety of inputs and outputs ensures safety across all data types.
• Project Astra and Mariner: Ongoing research focuses on preventing users from accidentally sharing sensitive information while providing privacy controls and ensuring user instructions take priority over malicious prompt injections.

Google continues to prioritize responsible AI development, ensuring safety and security are central to its approach.

AI Agents as Key Milestones Toward AGI: Google highlighted that the release of Gemini 2.0 Flash and several research prototypes marks a significant step forward in the Gemini era. These advancements represent an exciting milestone in AI development as Google works toward Artificial General Intelligence (AGI) while maintaining a strong focus on safety.

Availability:

• Gemini App Access: A chat-optimized version of Gemini 2.0 Flash is now available on desktop and mobile web, with the Gemini mobile app launching soon.
• Developer Access: Developers can access the experimental model through Google AI Studio and Vertex AI. Multimodal input and text output are available to all, with broader availability—including additional model sizes—expected in January.
• Multimodal Live API: A new API allows for real-time audio and video streaming, as well as combined tool use for developers.
• AI Overviews Expansion: Testing for complex tasks, including advanced math and coding, is underway, with a wider rollout planned for early 2025.

Discussing Gemini 2.0, Sundar Pichai, CEO of Google and Alphabet, stated:

“The advancements in Gemini 2.0 are the result of our decade-long investments in a unique, full-stack approach to AI innovation. Powered by custom hardware like Trillium and our sixth-generation TPUs, both training and inference for Gemini 2.0 were fully supported by TPUs. Trillium is now also available to customers for their own development.”

“While Gemini 1.0 focused on organizing and understanding information, Gemini 2.0 is designed to make that information much more useful. I’m excited to see where this next era takes us.”