OpenGlass introduces sensing-computing split architecture for local MLLM-driven visual assistance

Researchers present OpenGlass, an open-source system that splits sensing and computing to provide low-latency multimodal visual assistance for blind and low-vision users. The architecture uses an ESP32-based glasses unit to capture visual context while a nearby consumer device performs local MLLM inference, keeping raw data on user-controlled devices by default.

• Median user-to-audio latency reaches 993 ms with resized payloads and 1625 ms with raw 1280 x 720 payloads under real ESP32 Wi-Fi capture.
• 97.5% of trials fall below 2 seconds with resized payloads, while 93.3% do so with raw payloads.
• The system supports obstacle/hazard awareness, sign/object queries, and image-quality self-checking as a reference platform rather than a certified navigation aid.

OpenGlass reduces cloud reliance and network delays associated with cloud MLLM assistants by enabling local inference on wearable hardware.