eval-engineering

Eval Engineering Create one Harbor task at a time, run it, inspect the result, and repeat with the user. map harness + environment -> propose directions -> user chooses -> draft specs -> user approves -> build + run + audit -> repeat Use the latest Harbor release. Put task source under evals/ . Boundaries Task: instruction.md plus an Environment and Verifier. Harness: the complete agent Harbor runs: model, prompts, loop, repository-defined tools, middleware/hooks, memory/session behavior, and Harbor adapter. Harbor calls this the Agent. Environment: the container/world around the Harness: OS, files, backing data, services, identity, permissions, network, clock, and mutable state. Verifier: the test script that independently scores the response, trajectory, or resulting Environment state. Repository-defined tool code belongs to the Harness. The data or service behind it belongs to the Environment. Example: a docs agent's search_docs definition and result parsing stay in the Harness; the frozen search index and its error behavior live in the Environment. If production supplies a tool server dynamically, keep that server in the Environment and preserve how the Harness discovers and calls it. Show more Installs 889 Repository langchain-ai/la…n-skills GitHub Stars 1.1K First Seen Jul 22, 2026 Security Audits Gen Agent Trust Hub Pass Socket Pass Snyk Pass