eval-harness-first

Eval Harness First The Phase 0 gate for the whole plugin: finetuning-method-selection and every downstream skill assume this harness exists before a training config gets written. The harness is not a run-end side artifact — it is the data-curation engine. The same labeled traces that build the goldens feed training data, minus an explicit holdout. Input: production/agent traces if they exist, or a task spec if they don't, plus labelers willing to grade ≥100 examples. Output format: the eval/ directory below — goldens, graders, drift suite, and the base-model baseline that later phases gate on. The Gate Show more Installs 942 Repository wshobson/agents GitHub Stars 39.1K First Seen Jul 14, 2026 Security Audits Gen Agent Trust Hub Pass Socket Pass Snyk Pass