Harbor Framework

I’m a core member of Harbor, an open-source evaluation harness for agentic terminal tasks. Harbor gives researchers and engineers a reproducible way to run language-model agents inside real shell environments, capture their trajectories, and grade the results against task-specific checks.

What I work on

• Adapters — integrations that let Harbor evaluate agents and benchmarks from across the ecosystem through a single interface. This grew out of a collaboration with the Terminal-Bench team at Stanford on the Harbor adapters project.
• Core harness & runtime — the execution engine that spins up isolated task environments, drives agents through them, and collects structured results for grading and analysis.
• Maintenance — reviewing contributions, triaging issues, shaping the roadmap, and keeping the project healthy for a growing community of users.

Why it matters

As agents take on longer, multi-step terminal work, evaluation becomes the bottleneck: you need environments that are realistic, isolated, and reproducible, plus grading you can trust at scale. Harbor is built to make that kind of evaluation a first-class, shareable artifact rather than a pile of one-off scripts.

Explore the project on GitHub.