43 DAGs. 9 batch branches. One migration window.
Here’s what those numbers looked like before the architecture existed.
5–6 hours per DAG. Done manually by engineers carrying institutional knowledge that disappeared at the end of every sprint. A 15–20% defect escape rate, which means 1 in 6 DAGs shipping with a bug the review process missed. Documentation coverage at 30–40%: the migration decisions were being made correctly, and then forgotten. By the next engineer who touched the same codebase, by the next batch that ran against the same patterns, by the next sprint that inherited the same technical debt without knowing why it existed.
Not a rough draft. That’s a production migration with experienced engineers doing careful work.
Here’s what the system produced.
8–10 minutes per DAG. Documentation coverage at 100%, generated as a byproduct of the validation pass, not as a separate sprint. 43 DAGs across 9 batch branches. Every defect caught in a local sandbox before anything moved to a shared environment. The only issues that surfaced downstream were data problems, not migration defects.
That’s what zero escapes means here. Not that nothing ever broke. That nothing broke somewhere expensive.
The numbers are the proof. But the numbers only make sense once you understand what the architecture was catching.
Two bugs passed every static review layer. Both got caught.
The first: a YAML syntax error. Duplicate key. The Developer didn’t flag it. The Manager reviewed the code and cleared it. QA ran the command. The parser rejected it. Ninety seconds.
The second: an undocumented legacy pattern, a deprecated module reference that wasn’t in the transformation table. The Developer applied the standard transformation. At runtime, ModuleNotFoundError. QA flagged it. The rule got added to the skill document. Every subsequent DAG with that pattern was caught before it ran.
Neither is a model failure. Both are architectural catches. Static analysis tells you what code says it does. Runtime tells you what it actually does. When only one runs, there’s a guaranteed gap. Not in the model. In the structure.
The math across the full migration window: roughly five engineer-hours back per DAG, and a 15–20% defect escape rate going to zero. Across a batch of three, that’s about two working days. At 50 DAGs it’s around 250 hours, call it six weeks of one engineer, and a migration that runs 4–6 weeks faster.
None of this is free.
The skill document took multiple sessions to build: 18-pattern transformation table, prohibited patterns list, 40+ item validation checklist. The Manager orchestration layer required real engineering work. The Docker QA environment had to be configured and maintained. That work happened once. Every DAG that ran through the system after paid down the investment.
The demo-first pitch describes the best case, not the production case: agent count, speed claims, impressive output. Build the validation architecture first, or you’re shipping the demo. The system works because of the engineering that happened before it ran, not despite it.
Now the part I didn’t expect.
After every DAG the batch stopped and waited for me. I read the diffs in the editor, read the status document, read the error logs when there were any. I watched the Manager work while it worked, so I saw it catch the Developer’s mistakes and QA’s failures in something close to real time.
I did find things it missed. Not many, but some. And here’s what I did about almost all of them: I had the Manager confirm the problem and fix that DAG, and then I put the rule in the skill document.
Common mistakes. The checklist. The prohibited patterns.
Because the interesting thing about the misses is what caused them. It usually wasn’t the agent being wrong. It was a DAG written by one engineer in a house style that differed from another engineer’s, in a way the transformation rules hadn’t anticipated. A codebase with a decade of individual habits in it.
Fix the DAG and you’ve fixed one DAG. Write the rule and DAG 30 can’t reproduce it.
So the gate’s real output wasn’t caught bugs. It was a skill document that got harder to fool every week. The system that migrated DAG 40 was measurably better than the one that migrated DAG 4, and the thing that improved it was a human reading diffs and writing down what he learned.
Take the human layer out and you don’t just lose a safety net. You lose the only part of the system that was getting smarter.
The architecture is not migration-specific. But the precondition is easy to miss, and it’s the thing that decides whether any of this transfers.
Both bugs were caught mechanically. A parser rejected one. A runtime threw on the other. Neither catch required judgment. That’s what lets the layers stay independent: QA can return pass or fail without an opinion, so nothing has to be negotiated between agents.
Where nothing can return pass or fail on its own, the third layer isn’t a guard. It’s another opinion, and Episode 2 covered what happens then.
So the question for your domain is whether that check exists. Schema normalization across legacy systems has it: constraint violations and type checks decide it. Code review has it for correctness, and not for design. Compliance documentation has it for form and not for substance, which is the harder half and the one worth being careful about. Contract extraction mostly doesn’t, and I’d want to solve verification before pointing this at it.
The architecture is the same. The skill document is specific to the domain. Whether a machine can settle the question is what decides if either one matters.
The architecture solves the structural problem: validation integrity across a batch transformation where one agent can’t see the full scope of its own work. That problem is solved.
The second problem is the one the gate exposed. The skill document is what makes any individual agent reliable before it ever reaches validation, and it compounds everything downstream. A strong one means the Developer’s transformations are correct before QA sees them. A weak one means QA is doing remediation instead of validation. The knowledge layer is what the system runs on, and mine only got better because a person was sitting there writing it down.
That’s what the newsletter is on right now: the AI capital you’re building, and who actually owns it. Then what it takes to make that knowledge compound instead of reset every session.
The teams shipping zero escapes at scale didn’t find a better model. They decided validation was architecture, and built it before the agents ran.
Two paths from here.
If you’re the one accountable for a migration not shipping bugs: which of the four layers would break first in your stack, the Developer, the Manager, QA, or the human gate? Leave a comment with which one, and what it’s guarding now.
If you’re mapping this onto your own batch transformation: comment with the domain, and I’ll work through whether the mechanical check exists for it.