Graph problems and fixes

In this chapter, we will learn to diagnose a graph from its path and its node timings, as Chapter 8 did for loops from their traces. Seven problems cover nearly every bad graph run. For each: the symptom as the console or the trace shows it, the cause, and the fix in agl. By the end you will read a path like plan, write, review, write, review, write, review and know what is wrong before you read a single model output.

How to read a graph run

Three things tell you almost everything: the path, the per-node milliseconds, and the errors dictionary. The path is in state["_path"] and on the console's last line. A path that repeats a pair of nodes is a cycle that is not converging. A path that ends early is a stall. Per-node milliseconds show where the wall clock goes; errors show what died.

1. Good parts, bad whole: no merge node

symptom   every writer passes its own check; the reviewer fails the combined output every time, with the
          same complaint ("not one coherent piece", "repeats itself", "contradicts section 2")
cause     the goal needs information no single node has; the nodes that have it never share a wave or a prompt
fix       add the node that has it all: an editor between the fan-in and the review, fed every draft

This is the most common graph bug and no prompt fixes it. Three writers who never see each other cannot produce one article, however good each paragraph is. The feedback on a fail should go to the editor, with the previous version and the complaint, not back to all the writers: one call instead of three, and a better starting point. The general rule from Chapter 9: if the goal needs information that no single node has, add the node that has it.

2. A literal judge

symptom   the review fails at the same score every time (0.5, 0.5, 0.5) with feedback that quotes a clause
          of the rubric: "fails the 'one concrete example' constraint by introducing a second"
cause     the rubric says something you did not mean; the judge applies it exactly
fix       read the feedback, change the fewest words ("at least one"; "judge the whole"), test on graded answers

A consistent judge with a wrong rubric gives a stable wrong answer, which looks like a stubborn model. The score says fail; the feedback says why. Change the rubric, not the writers. Chapter 6 has the two habits that prevent this.

3. The graph stalls

symptom   the path ends early; errors is empty; a node that should have run never started
cause     a node is waiting for an input that cannot arrive: most often a repair edge counted as a dependency,
          or a fan-in whose branch was routed around
fix       check the back edges (g.back_edges()); for a branch a router may skip, do not fan in on it statically,
          route to the merge node instead

print(g.back_edges())                    # should list every edge that closes a cycle
s = g.run({}, plugins=Plugins(Console()))  # with stub nodes; read which node never started

Reproduce with stubs: functions that return their own name. The engine runs the whole shape in milliseconds and the path shows exactly where it stopped. Then look at the deps of the node that never ran. A stall is the one graph problem with no error message, because from the engine's point of view nothing went wrong; it is simply waiting.

4. A cycle that never converges

symptom   the path repeats (edit, review, edit, review, edit, review); the run ends with max_visits in errors
cause     the fail feedback does not change what the fixer does; or the goal is unreachable from here (see 1 and 2)
fix       make feedback specific and give the fixer the previous version; lower max_visits so it fails fast;
          add an escape: after two fails, route to a different fixer or to a human

g.router("review", lambda s: "pass" if s["ok"] else ("escalate" if s.get("fails", 0) >= 2 else "fail"),
         {"pass": END, "fail": "edit", "escalate": "ask_human"})

A cycle needs a stopping rule other than the ceiling. Count fails in the state and route differently once they pile up: to a stronger model, to a different strategy, or to a node that writes the problem to a queue for a person. max_visits is the net, not the plan.

5. Two nodes on one key

symptom   StateCollision raised on the first run: "'a' and 'b' both wrote 'k'"
cause     two parallel branches given the same output name
fix       name outputs after the node that writes them; merge in a node after the fan-in

This is the one error the engine raises instead of recording, because no run of the graph can be trusted until it is fixed, and it is best found on the first stub run. It is also the one problem on this list that cannot happen in production if you ran the stubs.

6. A node dies

symptom   errors["node"] holds an exception text; downstream nodes never run; the path ends early
cause     a tool raised inside plain-function node code, a network error, a model refusal, a parse failure
fix       retries for transient errors (Node(retries=2)); on_error to a fallback node; a loop instead of a plain
          function wherever a model is involved, because loops never raise; checkpoint so a rerun resumes

g.node("fetch", fetch_prices, retries=2, on_error="use_cached")
state = g.run(init, checkpoint="data/report.json")      # a crash at node 4 of 6 resumes at node 4

Plain-function nodes should do small deterministic things: write a folder, run a subprocess, reshape a list. Anything that calls a model or the network belongs in a loop, which returns a status instead of raising, or behind retries. With a checkpoint, a dead node is a rerun away from done.

7. Judges cost more than workers

symptom   the per-node cost of review nodes exceeds the writers'; the judge reads the whole source on every call
cause     the judge's prompt carries everything the task carried
fix       give the judge the minimum it needs (the answer, the task's facts list, the summary), not the full source;
          run deterministic checks first so the judge reads fewer answers; use the judge model once at the end,
          a cheaper check per node

A judge that reads twenty thousand tokens of source material per chapter can be three quarters of a build's cost. Most of that is unnecessary: the outline already produced a facts list per chapter, and a judge that checks the chapter against its facts list and its own summary catches the same inverted definitions for a fraction of the tokens. Measure it before and after; Chapter 15 shows how.

Where to change, in order

1. shape        is there a node that holds everything the goal needs? are back edges really jumps?
2. routers      does every cycle have a stopping rule besides max_visits? does fail go to the cheapest fixer?
3. evaluators   read the fail feedback; fix the rubric with the fewest words; checks before judges
4. nodes        retries and on_error for the fragile ones; checkpoint for long runs; loops where models are
5. prompts      last, inside one node, the one sentence the trace says is missing

Priya's three runs

Priya builds the article graph from the whiteboard and it fails three times before it passes once, and each failure is a different row above. First run: three writers, a reviewer, no editor; the reviewer fails the stapled output three times (problem 1). She adds the editor and routes fails to it. Second run: the reviewer fails three times at exactly the same score, and the feedback names a clause of her rubric that said "one example" (problem 2). She changes it to "at least one" and tells the judge to judge the whole. Third run: pass on the first review. No writer prompt changed in any of the three. The shape and the rubric were the bugs, and the path and the feedback named them.

Try it yourself

Build the article graph from Chapter 9 with stub nodes and break it in each of the seven ways: delete the edit node, write a rubric with an unintended "exactly", add fix -> assemble and comment out the back-edge check, remove the stopping rule from a cycle, give two writers the same key, make a node raise, and give a stub judge a 20,000-character input. For each, write down what the path, the timings and the errors show before you fix it.

Common questions

How do I tell a stall from a graph that finished early on purpose?

A finished graph reached END through a router or has no more edges to follow; its path includes the last node you expected. A stall ends before a node that has unfired inputs. Print g.back_edges() and the deps of the missing node.

Should fail feedback go to the writers or the editor?

To the cheapest node that can act on it with the most context. Usually the editor, who has the previous version. Writers get feedback only when the complaint is about one section.

What is a good max_visits?

Three. Most fixable fails are fixed on the first retry; the third is the one that tells you the goal is unreachable from here. Use a router with a fail count for anything smarter.

Can I make the whole graph never raise?

Everything except StateCollision already never raises. Keep that one: it is a design bug and you want to know on the first stub run.

Why checkpoint at all if nodes have retries?

Retries handle a flaky call. Checkpoints handle a crash of the process, a deploy mid-run, or a bug you fix and want to resume past, without paying for the nodes that already succeeded.

Carry this

• Read the path, the per-node times and the errors. A repeating pair is a non-converging cycle; an early end with no error is a stall.
• If the whole is bad and the parts are good, add the node that sees all the parts.
• A judge failing at the same score with the same feedback is a rubric bug. Change the fewest words.
• Every cycle needs a stopping rule besides max_visits. Route fails to the cheapest fixer with the most context.
• Retries, on_error and checkpoints make a graph survive the real world. Loops never raise; plain nodes should stay small.

Check yourself

1. The path reads plan, write_a, write_b, write_c, review, write_a, write_b, write_c, review, and the reviewer complains of incoherence each time. Which problem, and the fix?

Answer

No merge node. Add an editor between the fan-in and the review, fed every draft, and route fails to the editor with the previous version and the feedback.

2. A graph ends after two nodes with an empty errors dictionary. What is happening and what do you check?

Answer

A stall: a node is waiting for an input that cannot arrive. Check g.back_edges() and the dependencies of the node that never ran; a repair edge counted as a dependency is the usual cause.

3. How do you cut a judge's cost without weakening it?

Answer

Give it the minimum it needs (the answer, the facts list, the summary) instead of the full source, run deterministic checks first so it reads fewer answers, and measure before and after.