The Autonomous Software Factory: Engineering with AI Agents
If your coding agent cannot run what it writes, you are not automating engineering.
You are generating review work for yourself.
The Trust Curve
- Most teams use AI as autocomplete.
- Human review remains the scaling bottleneck.
To scale agentic engineering, ascend the Trust Curve:
| Level | Mode | Agent Role | Verification Gate |
|---|---|---|---|
| Level 1 | Copilot | Suggests inline syntax | Human writes code, tests manually |
| Level 2 | Pair Programmer | Implements functions | Human reads diffs, runs unit tests |
| Level 3 | Junior Engineer | Builds multi-file features | Headless runtime checks, human reviews PR |
| Level 4 | Software Factory | Issues reproduction test, builds, proves fix | Green mutation tests, auto-mergeable PR |
Two hard rules:
- You cannot reach Level 4 with prompt engineering.
- You reach Level 4 by closing the runtime verification loop.
The Inverse Pyramid
When agents fail:
- Beginners edit the prompt.
- Experienced operators fix the harness.
[ 10% Prompts & Task Scope ]
[ 20% Verification Skills ]
[ 70% Codebase Constraints ]
1. Codebase Constraints (70%)
- Type Systems & Strict Schemas: Pydantic, TypeScript, Rust. If the compiler rejects bad states, the agent cannot hallucinate them.
- AST Linters (Ban Nuance): Do not argue with agents in PR reviews about deprecated patterns or missing boundaries. Write an AST linter rule (ESLint/Ruff) that fails the build. If an agent cannot make the mistake, it won't.
- The Weeded Garden: Agents mimic surrounding repository patterns. Dead code, loose types, and sloppy error handling are bug multipliers. Delete unused code before extending it.
2. Verification Skills (20%)
- Headless Runtime Drivers: Drive web via headless CDP (Playwright). Drive desktop via offscreen Qt (
QT_QPA_PLATFORM="offscreen") and OS process boundaries. - Sidecar Observers: Run memory-leak checks, network assertions (zero unauthorized outbound TCP sockets), and layout overflow guards.
- Self-Healing Loop: If the agent sees runtime errors directly, it fixes them before a human ever looks at the PR.
3. Prompts & Task Scope (10%)
- Narrow task definitions.
- Explicit definition of done.
- Prompts state intent; code and tests enforce truth.
How Agents Determine Necessary Tests
Before touching production code, the agent runs a four-step interrogation:
1. Observable Proof
What exact artifact proves completion?
- UI: QML object property change or DOM render.
- Data: Exact database row or filesystem write.
- Math: Strict zero balance delta (
Diff: RM 0.00).
2. Boundary Gates
What input conflicts halt execution?
- Missing headers.
- Inverted signs.
- Date-range gaps.
- Currency mismatches.
Every boundary condition becomes an explicit negative test.
3. Blast Radius
What breaks down the dependency tree?
- Check adjacent modules.
- Compare against working twin modules to preserve shared contracts.
- Flag shared state modifications.
4. Mutation Testing (Prove the Test)
A test that cannot fail is worse than no test.
- Mutate: Invert or break the guarded source logic.
- Verify RED: Run the test suite assert failure (
exit code != 0). - Revert: Restore correct logic assert pass (
exit code 0).
If a test stays green while the logic is broken, the test is fake.
Bottom Line
- Writing code is no longer the hard part of software engineering.
- Verification is.
The engineers who win in the agentic era won't be typing prompts into chat boxes.
They will be the architects who design the AST constraints and headless verification harnesses that let agents ship autonomously.