Live Gurbani Tracking: Benchmark and Reference System for Sikh Kirtan Captioning

The authors present a benchmark and reference system for the live captioning of Sikh Kirtan, which involves the continuous sung recitation of verses from the Sri Guru Granth Sahib Ji. Unlike open-vocabulary transcription, this task requires exact word-for-word matches from the canonical scripture to avoid religiously inappropriate misspellings.

• The benchmark v1 includes 12 evaluation cases derived from four hand-annotated recordings and three cold-start offsets, totaling approximately 57 minutes of scored audio.
• A reference system using a fine-tuned 120M IndicConformer achieves 57.9% overall frame accuracy on the hardest live blind variant, with an RTF of ~0.05 on one Apple Silicon core.
• The scoring method computes frame accuracy at 1s resolution with a 1s collar and gap-tolerant scoring, addressing the specific display accuracy needs of the task.

The benchmark, reference system, and a live deployment are released under permissive licenses to facilitate further improvements in this specialized domain.