StrongDM Software Factory
The only well-documented "dark factory" running on real production code. Three engineers on StrongDM's AI team set two rules in July 2025: humans don't write code and humans don't review it. People write specs and usage scenarios; agents write code, run harnesses, and iterate until the scenarios pass.
The trick worth stealing is the holdout: scenarios are kept away from the coding agents, so they can't game them the way they game their own tests (StrongDM's agents literally wrote "return true" early on). The weak spot is that an AI judge scores the results, so the same kind of model both builds and grades. It's also expensive by design: their CTO's rule of thumb is about $1,000 of tokens per engineer per day.
Good fit if: you can write very precise scenarios and can afford heavy token spend. Not a product you install; it's a way of working they wrote up.
https://law.stanford.edu/2026/02/08/built-by-agents-tested-by-agents-trusted-by-whom/
https://simonwillison.net/2026/Jan/28/the-five-levels/
#ai #agents #softwarefactory
#
The only well-documented "dark factory" running on real production code. Three engineers on StrongDM's AI team set two rules in July 2025: humans don't write code and humans don't review it. People write specs and usage scenarios; agents write code, run harnesses, and iterate until the scenarios pass.
The trick worth stealing is the holdout: scenarios are kept away from the coding agents, so they can't game them the way they game their own tests (StrongDM's agents literally wrote "return true" early on). The weak spot is that an AI judge scores the results, so the same kind of model both builds and grades. It's also expensive by design: their CTO's rule of thumb is about $1,000 of tokens per engineer per day.
Good fit if: you can write very precise scenarios and can afford heavy token spend. Not a product you install; it's a way of working they wrote up.
https://law.stanford.edu/2026/02/08/built-by-agents-tested-by-agents-trusted-by-whom/
https://simonwillison.net/2026/Jan/28/the-five-levels/
#ai #agents #softwarefactory