Skip to content

Google introduces live captioning and Lens improvements to Android Q

Google announced at its 2019 I/O developer conference that the upcoming version of Android, currently codenamed ‘Android Q’, will feature some significant accessibility improvements relating to the automated captioning of video and the addition of search to Google Lens.

Live Captioning

The Live Caption feature will allow users to download a video to their device and play it back with captions regardless as to whether the video was formally captioned or not. This makes use of similar technologies currently found in YouTube’s automated captioned service whereby Google scans a video and adds captions for you. The main difference here is that the ability to scan a video is built into Android Q, and the process appears to be relatively instantaneous once a video is downloaded to a device.

While the Live Captioning feature is focused primarily on pre-recorded videos, it has also been demonstrated with real-time video calls. This has the potential to improve the communication options for people who are Deaf or hearing impaired worldwide. The following YouTube video showcases the feature in action.

While the promise of every video featuring captions and even live calling is extremely exciting for people who are Deaf or hearing impaired, there are currently few options to test the feature at this time of writing outside of specific beta testing programmes. There is also some scepticism about its accuracy given that the effectiveness of the YouTube automated captions feature relies heavily on broadcast-level audio quality and only caters for a limited number of languages and accents.

While the feature is primarily focused on people with a hearing disability, it is likely to have wider benefits for people wanting to watch video content in noisy environments such as on a bus or plane.

In terms of availability, people using Pixel and recent mobile devices affiliated with the Android One programme are likely to receive the update before the end of the year. Once a device is updated to Android Q, the feature can be enabled in the device settings.

Google Lens

Another accessibility-related improvement is an update related to Google Lens. Google has incorporated search and some additional real-time functionality to help people interact with your environment by taking a photo.

According to Natt Garun from The Verge, “Google says Lens can search for exact dishes on a menu and surface photos of that dish based on Google Maps information to show you just how it looks before you order. You can also point the camera at the receipt to bring up a calculator that lets you add a tip then split the bill or at a sign in a foreign language to hear a text-to-speech translation.”

Google Lens remains a popular feature in Android for people who are blind or vision impaired as it allows for a person with a vision disability to take a photo and find out what is in the surrounding environment. The added functionality is likely to continue making Google Lens more useful.

Accessibility features aside, the one remaining mystery about Android Q is its name. Google traditionally names its android releases after sweet treats and in alphabetical order, but as there aren’t many desserts that start with ‘Q’ it will be interesting to see what choice Google makes.

For additional information on Google 2019 I/O announcements, visit The Verge website article.

Published inUncategorized