Google has begun rolling out powerful new AI capabilities to its Gemini Live assistant, enabling it to analyze a smartphone’s screen or live video feed and respond to user questions in real-time. The long-awaited features, part of Google’s broader “Project Astra” initiative, are now becoming available to Gemini Advanced subscribers through the Google One AI Premium plan.
A Google spokesperson confirmed the release of the new functions, which mark a significant advancement in real-time AI interaction on mobile devices. The screen-reading feature allows Gemini to interpret what’s currently displayed on a user’s phone screen—such as text, images, or apps—and provide immediate context or assistance based on that content.
Meanwhile, the live video capability lets users activate their smartphone cameras and ask Gemini questions about what it sees in real time. In Google’s demonstration, for instance, a user pointed their camera at freshly glazed pottery and asked Gemini to help select a matching paint color. The assistant provided suggestions on the spot, showcasing its ability to analyze visual inputs and offer personalized recommendations.
These features are designed to bring users closer to a frictionless, multi-modal AI experience, enabling seamless interaction through text, visuals, and voice. The rollout builds on the vision outlined in Project Astra—Google’s long-term goal of developing an AI assistant that can observe, understand, and assist in everyday contexts.
The launch of these features also signals Google’s growing lead in the AI assistant race, with competitors still catching up. Amazon is preparing a limited early release of its Alexa Plus upgrade, while Apple has delayed its enhanced Siri platform. Samsung continues to offer Bixby, though Gemini remains the default digital assistant on its Galaxy smartphones.
As Gemini becomes increasingly versatile and context-aware, the assistant is poised to redefine how users interact with their devices—turning phones into intelligent partners capable of responding to the world around them.
Stay tuned to DC Brief for further updates on this story and other technology developments.
