Then it got a task where the spec was wrong.

The executor did exactly what it was told, the reviewer correctly said no, and the executor went back and did the same wrong thing again. Only, slightly differently.. Until the turn budget ran out. \sigh

Nobody was confused. Every node did its job. The problem was that the only edge out of the reviewer went to the executor, because when I drew the thing, why would there be an edge back to the planner? The planner had already planned. That was step one. We were past that.

And that’s the bug, and it isn’t in any of the nodes.

A script terminates when it runs out of steps. An agent replans when it runs out of steps.

That’s the whole difference. Everything else is plumbing we’ve all built a dozen times: a queue, a worker pool, a state store, retries with backoff. If you’ve written a job runner, you have all the parts. What you’re missing is one loop boundary in a different place.

I want to be clear that the pipeline instinct isn’t wrong. It’s the correct first move. It’s legible, it’s cheap, and for most of what people call “agents”, it’s also the last move they should make. The pipeline stops being right at a specific moment, and it’s worth being able to name that moment rather than reaching for the loop (because loops are what serious people build).

The call graph is the problem

Here’s the reframe that took me an embarrassingly long time to wrap my head around: the agents never call each other.

That’s literally it - if there’s one thing you take away from this post, it’s that. Internalize it.

The Misnomer

“Agent-to-agent” is a misnomer that smuggles an architecture in through the name. It sounds like a protocol between peers, right?

Well, it’s not. It’s agent, then state, then agent. Nobody’s phone rings. We’re not passing notes in math class - each node reads from a shared state, writes an artifact back, and a dispatcher decides who runs next.

And thus, here we are . It’s not a DAG - it’s a work queue with a scheduler. Not so scary now, right? The only interesting difference is that one worker type is allowed to enqueue new work, and its enqueue decision is a model output instead of a branch you wrote at 2am on a Tuesday six weeks ago after a few beers (not unlike this post).

That’s what the “edges” are, by the way. Every edge in your DAG is a decision frozen at authoring time. if review_failed: retry() is a guess about a situation you had not yet seen, made by a person who had less information than the system will have at the moment it matters. My reviewer-to-executor edge was a guess that review failures mean the code is wrong. Usually true, yes. But .. not always true. And the DAG has no way to find out, because the guess is load-bearing structure.

Delete the edges and the decision moves to runtime, where the information actually is.

Four node types, and one rule for placing them

Four kinds of node: planner, executor, verifier, reviewer. Their signatures tell you most of what you need, and the rule for placing them is simple.

If a node’s output is a decision, it’s frontier.

If it’s an artifact with a deterministic check, it’s the cheap model.

Decisions aren’t testable. That’s not a limitation to engineer around, it’s the criterion. “Should we retry or decompose?” doesn’t have an assert. But “Does pytest tests/test_auth.py::test_expired_token exit zero?” does. Route on that property, and the split falls out of the architecture instead of being a cost optimization you bolted on afterward.

Planner: state -> [task]

The only node with agency. It reads the entire history and decides what happens next, which makes it the one place in the system where “it depends” is an acceptable answer.

Two things people get wrong here. It runs on empty queue, not on schedule, and not on failure. There’s no on_error: replan() hook, because replanning isn’t an error path, it’s the normal path that happens to also cover errors. And it doesn’t plan the whole job. It plans the next chunk, which is usually one task, occasionally three, and never seventeen. A planner that emits seventeen tasks has quietly turned back into a DAG author, except now the DAG is written by something with worse judgment than you and no ability to draw it on a whiteboard.

Use the best model you can afford. This is where the money goes.

Executor: task -> artifact

Takes one task, produces one artifact. Makes no decisions, routes nothing, and never gets asked what should happen next.

Cheapest model that can hold a tool loop without falling over, which is a lower bar than it was a year ago and is the entire subject of the follow-up post. I’m a huge fan recently of protoLabs' quant of Ornith 1.0 35B @FP8

Verifier: artifact -> verdict

The one I’d tattoo on people: the verifier is code. Sear this into your memory. It is not a model. Not a model with a really good prompt. Not a model with a really good prompt and structured output. It. Is not. A model!

The instant the verifier is a model, you’ve rebuilt self-assessment, and self-assessment is the thing the whole apparatus exists to avoid. Done means verified deterministically or it doesn’t mean anything. An executor that grades its own homework is a very expensive way to generate confident nonsense, and it will do it politely, in JSON, with a "confidence": 0.946.

What this looks like in practice is unglamorous, and that’s the point. A shell command and an expected exit code. A schema validation. A diff that has to touch a file in the allowed set. grep -c on a log. If you can’t write the check, you don’t have an acceptance criterion, you have a vibe, and the planner shouldn’t have emitted that task in the first place.

Reviewer: artifact -> judgment

Reads artifacts that already passed the gate and forms an opinion about them. Use a frontier model, because opinions are decisions wearing a different smoking jacket.

A judgment is not a verdict and it’s worth keeping the words apart. A verdict is mechanical: the test passed, the build is green, the schema validated. It’s a fact, and code produces it. A judgment is an opinion, and opinions come in flavors: the fix mutes the symptom instead of curing it, the abstraction won't survive the next feature, the code is technically correct and morally a crime. Both are useful. Only one of them is allowed to gate anything.

The reviewer doesn’t route either. It records, same as everyone else, and the planner reads the record on the next tick and decides whether “morally a crime” is worth another three tasks or whether we ship it and open an issue.

The loop

Take a look at this conceptual python.

def run(goal, budget):
    state = State(goal=goal)
    queue = deque()

    while not goal_satisfied(state):
        if budget.exhausted():
            raise BudgetExhausted(state.summary())

        if not queue:
            tasks = plan_next_tasks(state)
            if not tasks:
                raise NoViablePlan(state.summary())
            queue.extend(tasks)

        task = queue.popleft()
        worker = worker_for(task.capability)
        artifact = worker.run(task, state.context_for(task))
        verdict = verify(artifact, task.acceptance)

        state.record(task, artifact, verdict)
        budget.charge(worker.cost)

    return state

Now look at what isn’t there.

There are no edges. There’s no if verdict.failed: retry(). There’s no routing table, no conditional handoff, no state machine enum with AWAITING_REVIEW in it. When verify fails, that just gets recorded. The queue drains. The loop comes back around, finds the queue empty, and asks the planner what to do about a state that now contains a failed verification. The planner might retry it. It might decompose it into three smaller tasks. It might notice the spec was wrong, which is the thing my DAG couldn’t do at any price.

The retry loop is emergent, not coded. That’s the self-driving property, and it costs exactly one if not queue:.

I find this genuinely hard to hold onto. Every time I read this loop I want to add an edge. Some case comes to mind, the reviewer rejects on style versus correctness and surely those should route differently, and my hand goes to the keyboard. Don’t. That’s a decision, decisions are the planner’s job, and the planner has strictly more information at tick 12 than you have right now.

State is the architecture

Two properties, and both of them are load-bearing.

Append-only, not a mutable blackboard. The planner’s next decision is a function of history. Overwriting is amnesia. If tick 4 recorded “tried the obvious fix, test still red” and tick 9 overwrites it, the planner at tick 10 will happily try the obvious fix. You also get replay for free, which matters the first time one of them does something whack at 3am and you need to know whether the planner was wrong or the state was.

context_for(task) is a projection, not a dump. Each worker sees what its job needs. The executor gets the spec, the file paths, the test command. It does not get the planner’s reasoning, the reviewer’s prior judgments, or the transcripts of its five siblings.

This is where most implementations die, and it dies in disguise. You dump full state into every call because it’s easy and it’s correct-ish, and by tick four you’re feeding the executor a novel. Quality degrades. It feels like a model problem. You go swap models. It’s not a model problem, it’s that you handed a worker fifty thousand tokens of other people’s thinking and asked it to find the one line that was actually its assignment.

Stopping

Three mechanisms. You need all of them, and they fail differently.

goal_satisfied is a predicate over recorded verifications. Never a model announcing success. Same reasoning as the verifier, higher stakes, because this one’s the exit.

Budget exhaustion raises, and it raises with a state summary, not just a log line. Something a human can read to find out what happened.

No-progress detection: if the state hash hasn’t moved in N ticks, kill it and escalate. This is the one people skip and it’s the one that saves you money, because a planner stuck in a loop of semantically-identical tasks will happily burn every token you’ll give it while producing state that’s byte-identical tick over tick.

The principle underneath all three: an agent that can’t finish has to say so loudly. Silent success is the worst failure mode you can ship, because it’s the one that ships. A crash gets fixed on Tuesday. A green checkmark on a job that quietly did nothing gets found in November.

Watching it work: incident to patch

Alert fires. Goal: figure out what borked and fix it.

Tick 1. Queue’s empty, so the planner runs. It reads the alert, the logs, and a metrics window, and emits one task: write a failing test that demonstrates the reported behavior. Not “fix the bug.” The planner doesn’t know what the bug is yet and it isn’t pretending to.

Tick 2. Executor takes it. Reads the code, writes a test, runs it. Artifact: a diff containing one new test. Verifier is a shell command: does the suite fail, and does it fail in the new test rather than somewhere else? Say it passes instead, meaning the test went green immediately. Recorded as a failed verification. Not an error. A fact.

Tick 3. Queue’s empty again, planner runs again, and now it’s reading a state that says “our first theory of the bug produced a test that passes.” That’s information. It emits a different task: check whether the alert is even real, or whether the metric threshold moved.

The DAG I would have drawn cannot do tick 3. It would have taken the passing test as a green light and gone on to patch a bug that wasn’t there.

Note the gate sitting between the repro executor and the patch executor. Two executors back to back with nothing in between compounds error: the second one builds on the first one’s work with no independent check that the first one’s work was real. The verifier isn’t ceremony, it’s the thing that keeps the second executor from inheriting a hallucination as a premise.

Run the same alert twice and you’ll get different task sequences, because the planner sees different state. That’s the payoff and it’s also the part that makes these systems annoying to test, which is a real cost and I’m not going to pretend otherwise.

When you shouldn’t build this

Most of the time. Despite the echo chamber of modern .. life.

If the task shape is known, the steps are fixed, and failure means stop, just write the damn script. Tell your director to shove it and punch them in the face. It’ll be faster, it’ll cost less, you can reason about it, and when it breaks you’ll know why in about ninety seconds. Nobody has ever been paged at 3am because their bash script had insufficient agency, or it hallucinated a glob as rm -rf / in a privileged environment.

The loop earns its complexity in exactly one condition: the task space is open and partial failure is expected and informative. Open means you cannot enumerate the paths ahead of time. Informative means a failure at tick 2 should change what happens at tick 3. If failures don’t change the plan, you don’t need a planner, you need a retry decorator and a good night’s sleep.

Most things sold as agents are scripts wearing a hat. That’s frequently correct. The hat is the problem, not the script.

The tell

Here’s how I know now, before I write anything: I ask what happens when the plan is wrong.

If the answer is “it fails and I look at it,” write the script. Genuinely. That’s a fine answer and it’s the right answer more often than the discourse suggests.

If the answer is “it needs to notice and try something else,” you don’t want a better DAG. You want a loop with a planner in it, and you want the courage to leave the edges out, which is harder than it sounds because the edges feel like engineering and the empty space feels like you forgot something.

You didn’t. That’s the design.