What to know
- Google’s Pixel 11 can translate American Sign Language into written English in real time.
- The feature works through Gboard and Live Transcribe, allowing users to sign messages, searches, notes and casual conversations.
- The Pixel 11 hits Canadian shelves on Aug. 20 and starts at $1,249, with pre-orders available now.
- ASL-to-English is the only option at launch, and Google says the tool should not replace qualified interpreters in high-stakes settings.
Google’s newest Pixel phone will let users communicate through American Sign Language without having to type.
The company has introduced a new sign-to-text feature for its Pixel 11 phone that can translate American Sign Language (ASL) into written English in real time. Powered by Google DeepMind’s new sign-language-to-text-model, known as SL2T, the accessibility tool will be available through Gboard and the Live Transcribe app.
Similar to voice dictation, users can position themselves in front of the phone’s camera and sign naturally. The translated words will then appear in the text field as they sign.
The feature can be used anywhere Gboard is accepted, allowing people to sign messages, write notes, search the web, create documents or submit prompts to Gemini. Users can also switch between signing and typing to review or correct the generated text before sending it.
In Live Transcribe, the technology is intended to make casual face-to-face conversations easier. A Deaf or hard-of-hearing user could, for example, read captions of what another person is saying and sign their response rather than type it.
Google says the feature was developed alongside Deaf community members, advocacy organizations, researchers and interpreters.
The company estimates there are more than 200 sign languages worldwide by approximately 70 million Deaf and hard-of-hearing people. However, sign-language technology has traditionally developed more slowly than tools for spoken languages.
ASL is its own language with a distinct grammar and vocabulary. Meaning can be communicated simultaneously through a person’s hands, arms, face, head and upper body, making translation more complex than matching individual signs with English words.
How Google’s sign-to-text technology works
According to Google Deepmind, the SL2T model was trained using more than 100,000 hours of data across over 50 sign languages. Approximately one-quarter of the data was in ASL.
When someone signs, an on-device model tracks points across their face, hands and upper body. Only the resulting geometric coordinates are sent to Google’s servers for translation, rather than the raw camera footage,
The model was also trained to recognize left-handed signers and one-handed signing, which may be necessary when someone is holding their phone with the other hand.
However, the technology is not intended to replace a qualified interpreter. A joint impact report published by Google and its AI Sign Language Advisory Committee says the current version is meant for low-stake situations, including messaging, online searches, note-taking and casual conversations.
Google also acknowledges that the model can still struggle with some regional sign variations, grammatical details, facial expressions, rare signs and rapid fingerspelling.
At launch, the feature will translate only ASL into English. Google says more sign languages and devices will be supported in the future but has not provided a timeline.
Online users praise Google’s accessibility push
The announcement has received a largely positive response online, with several Reddit users giving credit to Google for applying artificial intelligence to accessibility.
“I’m honestly surprised this sort of tech did not exist earlier,” one user on Reddit wrote.
“This is finally something great for humanity,” another person said.
Others said the feature is an example of how artificial intelligence can be used to make a meaningful difference in people’s lives.
“This is what AI should be used for. Actual improvement of human society,” one user commented.
Another person hoped the technology would eventually expand beyond Google’s newest phone.
“This is actually really awesome, imagine if this could be implemented into other apps. I wouldn’t know if something like this already exists but it seems good for deaf and or mute people,” the user commented.
When will Pixel 11 be released in Canada?
The Pixel 11 is currently available for pre-order in Canada and will officially hit shelves on Aug. 20 through the Google Store and participating retailers.
The base Pixel 11 starts at $1,249 in Canada, according to the Canadian Google Store.
Google’s support page says Canadian users can access sign-to-text by setting Gboard’s keyboard language to either English (United States) or English (Canada).
The feature will be included on Pixel 11 devices at no additional cost. Google says it plans to bring the technology to more devices, although it has not identified which phones will receive it next.
