BILAL-BENCH Quantifies Islamic-Source Fabrication in Bengali Language Models
November 2, 2026
BILAL-BENCH is a 240-question expert-verified benchmark for evaluating Islamic-source fabrication in Bengali. Testing seven models (1B to 120B) showed fabrication rates between 79.3% and 95.3% on source-production questions, with no significant improvement observed as scale increased.
HOW THIS AFFECTS YOU
●
researcherYou can use this to study how scale fails to mitigate hallucination in low-resource religious contexts.
●
policyThis demonstrates the high risk of authoritative-sounding misinformation in specific cultural and linguistic domains.