BiasShades Marathi (LLM Bias Evaluation)

BiasShades Marathi (LLM Bias Evaluation)

728 stereotypes with contrasts in parallel across 16 languages including Marathi. Annotated with regional and demographic features for evaluating LLM bias. The only bias/fairness evaluation dataset available in Marathi, critical for responsible AI development.

Build a fairness auditing tool for Marathi NLP models that measures bias across caste, religion, and gender dimensions.
HomepageHuggingFace

Quick Start

# BiasShades Marathi fairness benchmark
from datasets import load_dataset
print('BiasShades Marathi Fairness Dataset')
print('Evaluates NLP model bias across demographic attributes')
Modality
text
Size
300+ stereotypes from 37 regions, translated to 16 languages
License
Format
JSON / Parquet
Language
mr, en
Update Frequency
static
Organization
LanguageShades
Category
benchmarks-fairness-dialects-tools

Schema

FieldTypeDescription
textstringTest sentence with demographic attributes
attributestringDemographic attribute being tested
expected_labelstringExpected unbiased label

Build With This

Create an automated bias testing pipeline that evaluates new Marathi models before deployment for harmful stereotypes
Develop a debiasing technique for Marathi language models that reduces stereotypical associations while maintaining performance
Build a fairness leaderboard for Marathi NLP models that publicly tracks bias metrics alongside accuracy scores

AI Use Cases

LLM bias evaluation in MarathiFairness testing for AI systemsCultural stereotype detectionResponsible AI auditing
Last verified: 2026-03-09