Building a Vulnerability Discovery Harness

About this episode

Dan Jones didn't set out to build a vulnerability factory—he set out to hack Cloudflare more effectively. Late last year, sitting on the red team, he and a few colleagues clocked what was coming: They were about to receive hundreds of thousands of LLM-generated bug reports, and the only sane response was to get there first. They started building six months before Mythos existed, with no best practice to copy and everyone quietly trying a different approach. Then Cloudflare became one of the first enterprises inside Project Glasswing, the classifiers came down, and the harness went to scale. In this episode, Dan tells Mehul the full arc: the 450-line security skill he open sourced, why you tell a model what to do and never what not to do, how a hunting model will happily stand up a fake web server to earn a green check mark, and why his validators never get to see the hunter's exploit code. He also gets specific about the machinery—four models hunting in parallel, mini-Cloudflares built on demand, every turn and emission written to a database so nothing is ever lost to a dead SSH session, 21,000 harness emissions boiled down to roughly 7,000 real bugs with over half already fixed—and closes on the uncomfortable half of the problem that most enterprises haven't thought about yet: finding bugs is the easy part. "It's not worth finding the bugs unless you can fix them."