Most conversations about AI and testing start with the model. Mine started with the opposite question: what does a test actually need to be worth, before I let anything, human or agent, ship it into a codebase I’m accountable for?
That question came from years spent as a QE Manager before I ever wore an AI-native hat. It’s the reason the agentic quality pipeline I designed and now run in daily production use wasn’t built as an AI project with some testing bolted on. It was built as a testing discipline that happens to be executed by AI.
What it does, at a high level
Given a functional specification, the pipeline autonomously generates a complete test suite, covering API, UI, mobile, performance and security, orchestrated across multiple AI coding agents working in concert rather than a single model doing everything. It doesn’t just produce tests that run; it produces tests wired directly into the same CI/CD safeguards a human-authored suite would go through, so nothing generated is exempt from the checks everything else has to clear.
The criteria it’s built on
None of that is trustworthy on its own. What makes it usable at scale, across 25+ teams, is a set of QE principles enforced on top of the generation, not left to the model’s judgment:
- Traceability first. Every generated test maps back to an explicit acceptance criteria, not to a guess at what “looks testable.” If a test can’t be traced to a requirement, it doesn’t ship.
- Risk-based coverage. Effort is weighted toward the areas with the highest business impact and the highest rate of change, the same prioritization I’d apply manually, not spread evenly across the codebase because evenly is easier to automate.
- Test pyramid discipline. Unit and API-level assertions form the backbone; UI and end-to-end tests are reserved for what genuinely requires a browser or a device, not generated by default because they’re the easiest thing to visualize.
- Maintainability as an output, not an afterthought. Generated tests follow the same structural patterns enforced in our manually authored suites, Page Object Model and Screenplay-style separation, SOLID structuring, so a generated test and a human-written one are indistinguishable six months later when someone has to maintain it.
- Validated fault detection, not just green checkmarks. A test that passes isn’t automatically a good test. Generated suites are held to the same mutation-testing standard I’d apply to any team’s coverage: if a test can’t catch a deliberately introduced defect, it doesn’t count as coverage.
- A human still holds the gate. Every generated case is reviewable and overridable by a QE before it reaches CI. I treat the agent as a fast, tireless junior tester, not an unsupervised authority, which is exactly how I’d onboard a junior engineer onto a team I was managing.
Why both hats had to be on the same head
An AI-native Delivery Manager without QE depth ships something that produces green checkmarks and calls it coverage. A QE Manager without delivery fluency never lets an agent near production test suites at this scale, the operational trust just isn’t there. This pipeline exists because both of those perspectives had to make the same call at once: is this a good test, and does it belong in how we deliver.
That discipline is also why the results attached to it are what they are. The AI doesn’t produce numbers. The criteria does. The AI just makes it possible to apply them at a scale no manual QE process could match.