IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning

Abhinav Joshi, Shounak Paul, Akshat Sharma, Pawan Goyal et al.

Authors & affiliations

Abhinav Joshi, Shounak Paul, Akshat Sharma, Pawan Goyal, Saptarshi Ghosh, Ashutosh Modi

Affiliations reported at publication:

IIT Kanpur
IIT Kharagpur

2024ACL 2024, Volume 1: Long Papers

What it adds

Brings eight Indian legal tasks into one benchmark, spanning document understanding, precedent retrieval, summarisation, prediction and translation.

Read with this limitation

Multilingual coverage applies to translation; bail prediction is Hindi and other tasks are English. Legal subdomains remain incomplete. The authors endorse research use only; predicting historical outcomes does not establish legally valid reasoning.

Jurisdiction tags: India
Language: 10 languages
English · Bengali · Hindi · Gujarati · Malayalam · Marathi · Telugu · Tamil · Punjabi · Oriya

Research artifacts

Resources linked by the authors.

Follow the source

Selected statements located in the linked primary sources. These results have not been independently reproduced here.

The benchmark combines eight tasks, including legal entity recognition, rhetorical roles, precedent retrieval, summarisation and legal translation.
3 IL-TUR: Legal-NLP Benchmark; Table 2
The translation task covers English and nine Indian languages; the limitations identify bail prediction as Hindi and the remaining tasks as English.
Limitations; language inventory in Appendix A.8

ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation

Both examine Indian court outcome prediction and explanations; IL-TUR also evaluates other legal document tasks.

LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain

Both organise legal NLP evaluation across languages, with different jurisdictions and task coverage.

Entry status, source edition & rights

Index entry: Source-linked summary, prepared with agent assistance. Independent scholarly review is pending.
Archive admission: This bibliographic entry does not confer admission to the versioned archive.
Edition inspected: 10.18653/v1/2024.acl-long.618
Reading coverage: ACL publisher v2 PDF: main-text task design and limitations, Table 2, and Appendix A.8 language inventory inspected. Publication affiliations recorded; author order preserved. The full appendix was not exhaustively audited. Linked task artifacts were inspected, not executed.
Rights: External references only. Papers and research artifacts remain with their original hosts; their own terms apply.
Entry updated: 2026-10-15

Source-linked summary · independent assessment pending.