AbdaullahAG and Somia Abufakher have released SemGuard, an open-source, four-layer semantic security gateway designed to protect large language models from prompt injections, jailbreaks, and privacy leaks in Arabic, Arabizi, and English.
- The system features a cascaded pipeline including Unicode normalization, a sub-millisecond regex fast-check filter, and a triple-anchor semantic classifier using multilingual-e5-large embeddings.
- SemGuard achieves a 0.992 F1-score on Arabic prompt attacks, outperforming baselines like ProtectAI by +21.3 F1 points while maintaining sub-millisecond latency.
- The authors also released the SemGuard Benchmark Dataset containing 807 human and LLM-validated examples across seven threat categories.
- A key finding from the dataset is that state-of-the-art LLMs disagree 98.2% of the time when distinguishing between harmful impersonation and benign roleplay in Arabic.
SemGuard provides an explainable alternative to black-box classifiers by outputting detailed JSON reports on blocked or allowed requests, addressing the security gap for under-resourced languages.