Real-time sentence-level sign language translation on Raspberry Pi

Researchers developed a real-time sentence-level sign language translation system by fine-tuning a SHuBERT-ByT5 stack on a 9,872-example subset of How2Sign. The model achieved a validation BLEU of 16.7 and test BLEU of 15.9.

The system utilizes a hardware-aware streaming architecture where a Raspberry Pi 4B handles capture and display while a backend performs perception and translation. Optimizations including chunked ingestion, bounded queues, and temporal reordering reduced mean post-finalization response latency from 1.873 to 1.354 seconds.

These improvements lower P95 latency from 2.919 to 2.130 seconds, enabling practical deployment for natural communication across various client devices.