Paul Iusztin released an open-source course 'Building a Coding Agent From Scratch' that demonstrates harness engineering trumps model choice. In LangChain's Terminal-Bench, swapping only the harness moved a coding agent from ~30th place into the top 5.
Key facts
- Harness swap moved agent from ~30th to top 5 on Terminal-Bench.
- Course builds Decode from scratch in Python.
- Final architecture: swarm of parallel remote agents in cloud.
- Course is fully open-source on GitHub.
- No model change was made between the two runs.
Paul Iusztin's new open-source course, "Building a Coding Agent From Scratch," makes a provocative claim: the harness, not the model, determines coding agent performance. According to @pauliusztin_, in LangChain's Terminal-Bench experiment, changing only the harness while keeping the same model lifted a coding agent from roughly 30th place into the top 5 — a 25+ position jump.
Terminal-Bench evaluates coding agents on terminal-based software engineering tasks. The harness controls tool orchestration, error recovery, and execution flow — the scaffolding around the LLM. The result suggests that current benchmarks may over-weight model capability while under-weighting infrastructure.
The course walks through building Decode, a complete coding agent from scratch in Python. It progresses from a bare agent loop to a swarm of remote agents running in parallel in the cloud. [The material is fully open-source on GitHub] and targets engineers who want to understand the infrastructure layer of agentic systems.
Key Takeaways
- Open-source course shows harness engineering, not model swap, moved a coding agent from ~30th to top 5 on Terminal-Bench.
- Course builds Decode from scratch.
Why the harness matters more than the model
The finding echoes a pattern seen in retrieval-augmented generation (RAG) systems: pipeline design often dominates model choice. For coding agents, the harness handles prompt construction, context window management, tool call formatting, and error retry logic — all places where naive implementations fail silently. A 25-place swing on a standardized benchmark with no model change is a strong signal that the community should invest in harness engineering.
What the course covers
Iusztin's course is not a theoretical survey. It builds Decode incrementally: starting with a single agent loop, adding tool dispatch, then scaling to multiple agents with remote execution. The final architecture is a swarm of agents running in parallel on cloud infrastructure. The focus is on production-grade patterns: retry logic, state management, and inter-agent communication.
The course is available now on GitHub. Iusztin did not disclose enrollment numbers or any associated pricing — the material is fully free.
What to watch
Watch for independent replication of the Terminal-Bench harness experiment by other teams. If the 25-position jump holds across different base models, expect a surge in open-source harness tooling and a shift in how coding agent benchmarks are reported — separating model score from harness score.
