IndicXTREME Benchmark

IndicXTREME Benchmark

MH Specific

Marathi — Comprehensive NLU benchmark of 9 tasks across 20 Indian languages including Marathi, covering classification, structure prediction, QA, and sentence retrieval

Evaluate Marathi language models on IndicXTREME's diverse task suite for comprehensive performance assessment.

Homepage HuggingFace GitHub

Quick Start

# IndicXTREME benchmark
from datasets import load_dataset
print('IndicXTREME: Comprehensive Indic NLP benchmark')
print('Multiple tasks including Marathi')

Modality

Text (Marathi)

Size

9 tasks, 105 evaluation sets

License

Open Research

Format

CSV/JSON

Language

mr

Update Frequency

static

Organization

AI4Bharat, IIT Madras

benchmarks-fairness-dialects-tools

Schema

Field	Type	Description
task	string	Evaluation task name
input	string	Task input text in Marathi
target	string	Expected output or label

Build With This

Create a multi-task Marathi model trained on all IndicXTREME tasks for a single deployable system

Develop a task difficulty ranker identifying which NLP tasks remain hardest for Marathi models

Build a cross-lingual transfer effectiveness study comparing Marathi performance with related Indic languages

AI Use Cases

Comprehensive Marathi NLU evaluationmodel comparison across tasks

Related Datasets

BiasShades Marathi (LLM Bias Evaluation)

FLORES-200 Benchmark

Text (parallel, Marathi)

Indic NLP Library

IndicGLUE Benchmark

Last verified: 2026-03-07