research paper · exact source version
Meta-Harness: End-to-End Optimization of Model Harnesses
v1 · Mar 30, 2026 · Legacy record · quality not assessed
- Contribution
- The paper reports a method for optimizing LLM harness code and includes a LawBench criminal-charge classification experiment in which the selected harness reaches 45.0% accuracy, compared with 29.0% for ACE in the authors’ setup.
- Limitation
- Legal evidence is limited to one LawBench criminal-charge classification benchmark within a broader three-domain AI-systems evaluation, and LegalArxiv has not independently reproduced any reported result.
Abstract
Source-authored
The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed largely by hand, and existing text optimizers are poorly matched to this setting because they compress feedback too aggressively: they are memoryless, condition only on scalar scores, or restrict feedback to short templates or summaries.
We introduce Meta-Harness, an outer-loop system that searches over harness code for LLM applications. It uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. On online text classification, Meta-Harness improves over a state-of-the-art context management system by 7.7 points while using 4× fewer context tokens.
On retrieval-augmented math reasoning, a single discovered harness improves accuracy on 200 IMO-level problems by 4.7 points on average across five held-out models. On agentic coding, discovered harnesses surpass the best hand-engineered baselines on TerminalBench-2. Together, these results show that richer access to prior experience can enable automated harness engineering.
Evidence map
1 mapped claim · v1
Follow a claim to its source passages, reported evidence and recorded checks.
Claim 1In the authors’ test setup, the selected Meta-Harness text-classification harness achieved 45.0% accuracy on LawBench, compared with 29.0% for ACE.
source authored · reported empirical result
Source passages
The places this claim is traced to.
Reported evidence
Section 4.1 describes LawBench as a 215-class criminal-charge prediction task; Table 2 reports 45.0 for Meta-Harness and 29.0 for ACE.
Access: public exact-version source HTML
Recorded check
The claim was checked against the exact-version source HTML; LegalArxiv did not rerun the benchmark.
Platform recheck: not performed
Limits of this claim
- LawBench is one of three text-classification datasets in the study.
- The claim is scoped to the authors’ model, harness, data, and evaluation setup.
- LegalArxiv has not independently reproduced the metric or audited the benchmark artifacts.
Contributors & provenance
- Evaluation status
- source reported not independently reproduced
- Recorded by
- actor:legalarxiv:importer
- As of
- 2026-09-01T09:23:41.000Z
- Claim identifier
claim:arxiv-2603-28052-v1:lawbench-accuracy- Exact version
version:arxiv-2603-28052-v1
Quality screening
Legacy record · quality not assessed
This record predates the evidence-based admission gate. Its presence in the archive does not establish that it passed the current quality policy.
Screening scope and limitations
- This record predates the quality gate and has no assessment under the current admission policy.
Inspect the current screening receipt and assessment references ↗
Detailed record status
Status remains facet-specific. Deposit, reconstruction, evaluation, peer review, currentness, and completeness are not compressed into one score.
- deposit
- source preprint v1The identified source is arXiv version 1, submitted 30 March 2026.
- archive record
- curated source indexLegalArxiv indexed the source and one legally relevant result; the authors have not attested this record.
- peer review
- accepted at colm 2026The work was listed as accepted at COLM 2026 when this record was prepared.
- independent evaluation
- not performedNumerical findings remain source-reported rather than independently verified by LegalArxiv.
Source sections · 10
- 1Source index
Introduction
Document-level author group; no section-specific byline stated
- 2Source index
Related Work
Document-level author group; no section-specific byline stated
- 3Source index
Meta-Harness: A Harness for Optimizing Harnesses
Document-level author group; no section-specific byline stated
- 4Source index
Experiments
Document-level author group; no section-specific byline stated
- 5Source index
Discussion
Document-level author group; no section-specific byline stated
- ASource index
Qualitative Proposer Behavior
Document-level author group; no section-specific byline stated
- BSource index
Discovered Harnesses
Document-level author group; no section-specific byline stated
- CSource index
Dataset Details
Document-level author group; no section-specific byline stated
- DSource index
Practical Implementation Tips
Document-level author group; no section-specific byline stated
- ESource index
Extended Related Work
Document-level author group; no section-specific byline stated
Source edition, rights & technical exports
- Source
- arxiv:2603.28052v1
- As of
- 2026-09-01T09:23:41.000Z
- Schema
- legalarxiv.record.v3
- Packet digest
- sha256:d687e58604a4720daeed820b90fd221786cef0fc487749c45b0d489ff02b757c
- Immutable packet
- Open digest-addressed representation
The source license and applicable law control; this field is descriptive.