Code review passed. Two agents approved it. The DAG would have failed on import anyway.
That’s not a model failure. That’s a structural one. It took a third agent, running a single command, to understand the difference.
We were three DAGs into a 43-DAG Airflow migration from Azure to Google Cloud. Each DAG moved through the same sequence: a Developer agent transformed the code, a Manager agent reviewed it independently, a QA agent ran it. The Developer and the QA agent spawned fresh each time: clean context, no memory of the agents before them, no assumption carried forward from prior runs.
DAG 3 was an orchestration file. More complex than the first two: more task dependencies, more operator configurations, more moving parts. The Developer wrote it and produced a review document listing every change it made. The Manager read that document and checked the code against the transformation rules directly. Task operators confirmed, dependency chains traced. High confidence. Approved.
Then the QA agent ran one command: airflow dags list-import-errors.
Error. YAML: duplicate key.
A duplicate key in a mapping node. The kind of thing you can read ten times and not catch, because nothing in the visual layout signals it’s wrong. But YAML parsers don’t read files. They process them. And a duplicate key is a hard failure: the DAG doesn’t load, doesn’t run, doesn’t exist as far as Airflow is concerned.
The fix took 90 seconds. The QA agent identified the error, the Manager confirmed it and corrected it, the command re-ran clean. The batch moved forward.
Two agents approved that configuration. One of them exists specifically to review the other’s work. Both did exactly what they were designed to do. Neither of them could have caught this.
The Developer wrote code. The Manager read code. Both doing the same kind of work: static analysis. Reading the file, checking structure and logic against a set of expectations. What neither of them did, what neither of them could do inside their role, was run it.
YAML syntax errors don’t exist in the reading. They exist in the parsing.
You can read a duplicate key and not see it. The parser sees it immediately. The Manager, running the most careful static analysis possible, is structurally incapable of catching a class of errors that only appear under execution. More review of the same kind wouldn’t have changed the outcome. More diligence wouldn’t have changed the outcome. Only a different mode of verification could.
The QA agent isn’t a better reviewer. It does a different thing: runtime validation. One command: load the DAG, report what Airflow actually processes. Not what the code says. What the system sees.
There are two modes of verification, and most processes use only one.
Static analysis is fast, scalable, good at logical errors, structural problems, and known anti-patterns. It evaluates what’s visible in the code. It has a ceiling: it can only see what exists before execution.
Runtime validation runs the output. Parse errors, import failures, configuration problems that only appear when the system actually processes the file. An entire category of error that static review cannot see, structurally, ever.
When both modes run, they cover each other’s blind spots. When only one runs, there’s a guaranteed gap. Not in the model. In the architecture.
The alternate timeline isn’t hypothetical. A YAML error in an Airflow DAG takes the entire batch offline. With 43 DAGs queued, a failed batch means re-queuing, re-running, emergency triage inside a migration window that doesn’t have slack in it. Everything here was caught in a local sandbox, before any of it reached a shared environment. That was the point of running it this way.
“Code review passed” is the most useless thing anyone can say in that moment.
The architecture is built around that blind spot. Not because any individual agent is weak, but because the structure provides what no single agent can: two modes of verification, running independently, neither able to inherit the assumptions of the one before it.
The QA agent doesn’t know the Developer wrote the file. It doesn’t know the Manager approved it. It runs the output and reports what the system sees. That independence is the feature. Give it the prior review and you introduce confirmation bias: the tendency to find what the last pass found and miss what it missed.
Deliberate distrust is the design.
Static analysis shows intent. Runtime shows truth.
There’s a layer above all three. After every DAG, the whole thing stops and waits for a human to say continue. That gate is not a formality, and what it produced was not mostly bug catches. Episode 2 is the architecture underneath it, including the version of it we built first and had to throw away.
One question before then: what does your review process actually run, and what does it only read?
Comment with what you’re running. I read every one.