ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts
Findings of EMNLP 2021, 1907–1919
What it adds
Pairs document-level entailment judgments with supporting evidence spans.
Read with this limitation
Fixed hypotheses across train/test and NDA-only coverage limit claims about unseen contract questions; label imbalance hides weak contradiction detection.
- Jurisdiction tags
- United States
- Language
- English
Follow the source
Selected statements located in the linked primary sources. These results have not been independently reproduced here.
The dataset comprises 607 NDAs annotated against 17 hypotheses.
2.2–2.3; Table 1, printed 1909Span NLI BERT-large reports 87.5% accuracy but only .357 contradiction F1.
4.2–4.3, printed 1912
Connected research
Cites
CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review
Compares extraction with inference; tests CUAD-fine-tuned models.
Inspect the sourceRelated question
MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement Understanding
Both test interpretation of contractual text through structured questions, using different agreement types and answer formats.
Entry status, source edition & rights
- Index entry
- Source-linked summary, prepared with agent assistance. Independent scholarly review is pending.
- Archive admission
- This bibliographic entry does not confer admission to the versioned archive.
- Edition inspected
- 10.18653/v1/2021.findings-emnlp.164
- Reading coverage
- Sections 2–5, Tables 1–4, evidence task definition and transfer experiments; official data landing page and code README.
- Rights
- External references only. Papers and research artifacts remain with their original hosts; their own terms apply.
- Entry updated
- 2026-10-15
Source-linked summary · independent assessment pending.