Google's Guided Vision adds real-time audio descriptions to Gemini Live

Google has started rolling out Guided Vision, a new feature in Gemini Live that gives real-time audio descriptions of whatever a user points the phone's camera at. It works on compatible Android devices and is designed for people who are blind, have low vision, or want assistance in specific situations.

What Guided Vision does

With the camera shared in Gemini Live, a user can point the phone at text, objects, or their surroundings, and Google's AI describes what it sees out loud. Google says Guided Vision can help with reading small text, describing surroundings, finding or identifying objects around you, and describing details on specific objects.

Users can also ask follow-up questions about what they are looking at, such as asking Gemini to read the expiration date on an item it just helped them find. If someone asks about something that is not in the shot, Gemini provides audio cues to help line up the camera.

Where it is available

Besides the Gemini app, Guided Vision is available in Google TalkBack, and users can set up an accessibility shortcut for it in the Settings app on devices running Android 9 or newer. The feature was announced as part of a recent batch of Pixel updates.

Limits and Apple's similar feature

Guided Vision is similar to VoiceOver Live Recognition, an accessibility feature Apple has added to the iPhone and Vision Pro. Google cautions that users should not rely on Guided Vision for navigation, safe-travel guidance, or obstacle detection, and that it is not a replacement for a cane or a mobility aid.