LawFlow: Collecting and Simulating Lawyers’ Thought Processes on Business Formation Case Studies
COLM 2025; inspected arXiv v2
What it adds
Records how trained law students plan, revise and carry out simulated business-formation work. Comparing these traces with model-generated workflows exposes differences that final-answer scores can miss.
Read with this limitation
Only ten finalized scenarios, with students role-playing lawyers and clients. Structured task plans and prompting can shape the observed differences. Logged actions reveal external behaviour rather than the models' internal reasoning.
- Jurisdiction tags
- United States
- Language
- English
Follow the source
Selected statements located in the linked primary sources. These results have not been independently reproduced here.
The final dataset contains ten scenarios, ten memos and eight agreements.
3. Dataset of Legal WorkflowsThe authors caution that more linear model workflows may partly reflect the prompting setup.
6. Conclusion
Connected research
Cites
LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
Uses LegalBench when motivating the gap between isolated tasks and workflow evaluation.
Inspect the sourceEntry status, source edition & rights
- Index entry
- Source-linked summary, prepared with agent assistance. Independent scholarly review is pending.
- Archive admission
- This bibliographic entry does not confer admission to the versioned archive.
- Edition inspected
- 2504.18942v2
- Reading coverage
- Exact arXiv v2 full HTML inspected and retained privately; sections 3, 4, 6, 7 and references. Conference acceptance independently listed by COLM.
- Rights
- External links only; no full text mirrored on the Site. arXiv labels this version CC BY 4.0.
- Entry updated
- 2026-10-15
Source-linked summary · independent assessment pending.