Google DeepMind and Android have introduced SL2T, a massively multilingual sign-language-to-text (SL2T) translation model that brings sign language AI to consumer products for the first time. The technology powers sign-to-text dictation in Gboard and Live Transcribe on Pixel 11, starting with American Sign Language (ASL) to English.

  • SL2T is trained on over 100,000 hours of data across more than 50 sign languages, achieving a zero-shot score of 70 BLEURT on FLEURS-ASL benchmarks.
  • The model translates directly from pose landmark coordinates tracked by MediaPipe Holistic, bypassing intermediate glosses to capture non-manual markers and spatial constructions.
  • Privacy is maintained by sending only geometric coordinates to the server, allowing original video data to be discarded immediately.
  • The system addresses practical usability issues such as streaming latency, hallucination prevention, and performance for left-handed or one-handed signers.

The release aims to bridge the communication gap between Deaf and hearing communities by enabling faster, more natural interaction on mobile devices. Development was guided by the AI Sign Language Advisory Committee (AISLAC) to ensure responsible deployment and cultural sensitivity.