GSM-Plus-BN introduces perturbation-based benchmark for Bangla math reasoning

The study introduces GSM-Plus-BN, a novel perturbed Bengali mathematical dataset derived from the English GSM-Plus benchmark and verified by human translators. This resource addresses the lack of systematic benchmarks for assessing model robustness in linguistically diverse regions like Bangladesh.

• The benchmark comprises 9,000 evaluation samples, including 1,000 seed questions and 8,000 perturbed variants.
• Six open-source LLMs were evaluated: Qwen3-32B, Llama-3.1-8B-Instant, Llama-3.3-70B-Versatile, Llama-4-Scout-17B-16E-Instruct, GPT-OSS-120B, and GPT-OSS-20B.
• GPT-OSS-20B achieved the highest seed question accuracy of 96.08% under Standard Prompting.
• Larger models like Llama-3.3-70B and GPT-OSS-120B demonstrated superior robustness across perturbation types.
• Chain-of-Thought prompting substantially improved reasoning for most models compared to Standard Prompting.

This research provides a foundational resource and baseline for future Bengali mathematical reasoning research, highlighting the inherent difficulty of perturbed Bengali text.