Documentation perpetually being chased, authorization processes lagging years behind the environments they evaluate, compliance teams operating in a separate lane from the engineers building the systems they are supposed to protect. These are the compliance realities most organizations recognize, and for a long time, they were accepted as normal. Then SolarWinds happened.

"SolarWinds was really a wake-up call," said Bob Costello, former Chief Information Officer at the Cybersecurity and Infrastructure Security Agency, at the Secureframe National Cybersecurity Summit last week. "It changed the conversation from compliance-based security to actually understanding the risk posture of your entire environment."

Costello spent nearly five years as CISA CIO, where he was responsible for the information systems and mission technology that powered the agency's cybersecurity work across the federal civilian government, including the analytics platforms, protective services, and security tooling that CISA used to monitor and defend federal networks. Before that, he held senior leadership roles at U.S. Customs and Border Protection and Immigration and Customs Enforcement and spent nine years in the U.S. Air Force supporting communications and intelligence systems. 

What he found when he arrived at CISA led him to rebuild the authorization process from the ground up, cutting the time to achieve an ATO from over a year to roughly 35 days and piloting AI-assisted penetration testing in active production environments.

His conclusion? The old compliance model is not just inefficient, it is a liability. And AI, used correctly, offers the first real path to closing the gap between compliance and actual security.

The 3-year ATO is a gift to US adversaries 

When Costello arrived at CISA, he found systems waiting three or more years for authorization. Security control assessors were testing controls once and not returning for years. Sections of System Security Plans (SSP) were cut and pasted from prior versions, sometimes still referencing the wrong system entirely. "I'd be reading a document," Costello recalled, "and I'd come to a place and think, 'They didn't really edit their copy and paste. It's talking about another system now.’"

The problem ran deeper than documentation. Costello remembers, “You would get something to sign as an authorizing official and say, 'The risk is moderate.' Well, how do we know?" The compliance record of a system and its actual security posture had quietly diverged, sometimes by years, and nobody had a reliable way to see it. 

The adversary, meanwhile, was operating on a completely different timeline. “They’re constantly testing us,” Costello said. “They get in, they wait years, and they’re very hard to detect. They’re playing the long game.” A compliance model built around annual reviews and three-year cycles is facing an adversary that operates continuously, adapts in real time, and has demonstrated the ability to live undetected inside critical systems for months. “With the pace of development today and the rapid changes that the adversary brings to their tactics,” Costello said, “that doesn’t work anymore.” 

When the SolarWinds attack compromised the software supply chain, it introduced a vulnerability into widely-used code and used it to pivot silently through environments that had passed every compliance check. Part of what made it so damaging was how deeply the attack embedded itself in the identity systems that control who and what can access everything else. 

"We actually were running SolarWinds on an isolated network where we couldn't just shut it off because that system was providing essentially radio services to law enforcement offices, so we couldn't stop monitoring it," Costello recalls.

That experience stuck with him. Attackers were no longer primarily targeting network boundaries. They were targeting the human and machine identities that controlled access to everything inside them. “Adversaries are very interested in compromising the identity of people and machines, and we had to really up our game,” says Costello. 

How AI turns compliance into a continuous practice

Compliance automation tools have been around for years, including automated vulnerability scanners, GRC platforms, and patch management systems. But even the best tooling can only do so much when underlying compliance models are built around fixed schedules and periodic reviews. The problem is the cadence, not the capability, and AI is what finally makes continuous operation practical at scale. 

What AI changes is the ability to run those same tools at the speed of the threat environment rather than the speed of the compliance calendar. AI pen testing can run hourly, daily, or automatically after every deployment. AI-assisted monitoring can surface patterns across thousands of alerts that no human analyst team could process in real time. 

"You're sitting in the SOC. You're getting thousands of alerts a day. How do you actually know what's important, what's not? AI can help a lot there," Costello said. AI-enhanced documentation workflows can eliminate the weeks of manual work that kept authorization timelines measured in months. 

"The other area where AI can help us is testing of controls and doing it as often as possible, which brings a higher level of assurance. And you need to test controls in production because that's what adversaries are doing." Controls can be tested continuously, risks can be reassessed continuously, and security teams can validate fixes almost immediately after deployment instead of waiting for the next audit cycle.

But none of this works if the underlying compliance model stays periodic. In Costello’s view, the organizations getting results are the ones pairing AI capabilities with a fundamentally different approach that doesn’t treat certification as the finish line. “Right now, we’re really seeing the point where compliance is being viewed not as the team of people that only talk to you when something went wrong, but as a team that has to be brought in from the start of a project to make sure it’s successful.” 

The AI pen test that surfaced an issue in seconds

Shortly after CISA deployed its AI penetration testing pilot, it found an unnoticed vulnerability.

“We rolled it out in an environment and found something that no one ever wants to find: an exposed management interface of our firewall exposed to the internet. The other tools hadn’t picked it up. The humans hadn’t recognized it. We identified it within seconds,” Costello explains. 

An exposed firewall management interface is the kind of vulnerability that can give an attacker direct access to network infrastructure. It’s also the kind of issue that a human pen tester, working within the constraints of a scoped engagement and fixed number of hours, might never surface. Not because they aren’t skilled, but because there simply isn’t enough time for that level of comprehensive coverage. “Often, the contract said you’d have 50 pen testers,” Costello noted, “but it was never fully staffed, and people would leave very quickly because they have very in-demand skills.”

AI doesn’t operate with those constraints. It can be triggered automatically after every deployment and can chain vulnerabilities together the way a sophisticated human attacker would, simulating multi-step attack paths that traditional automated scanning isn’t configured to find. 

“For a long time, the standards were to patch criticals within 24 hours, and highs within two weeks,” Costello said. “But what you’re really seeing now is the adversary chaining together three low vulnerabilities and getting into the system.” The AI tools CISA piloted could simulate exactly that kind of attack path, repeatedly and rapidly, across the entire environment rather than a sampled subset. 

Costello was also struck by what the tools did for the people watching them work. “It was an AI that showed you its work, what it did, how it did it, what it discovered, and you could audit it. It really helped make junior troops operate more like senior troops.” In a field perpetually short of experienced practitioners, an AI that closes that skill gap transparently and auditably changes what a small security team can realistically accomplish.

AI is changing the role of compliance teams

Costello spent nearly five years setting cybersecurity standards for the federal civilian government, shortened the ATO timeline 10x, piloted AI pen testing in production, and moved CISA entirely off Active Directory. Now, he’s navigating CMMC as a defense contractor at Merlin. 

“I always watched CMMC from the stands,” he said. “Now it’s here.” 

The same dynamics he worked to fix at CISA are now showing up in his own organization’s compliance journey.  “We can’t just use Microsoft 365 Commercial anymore. We’re going to have to know who’s handling CUI. We’re migrating a lot of our environment to FedRAMP equivalent versions of the same tools. This is going to be almost a complete reset.” But rather than frame it as a complaint, Costello sees this as the right forcing function. “If you’re applying these controls, you actually reduce your exposure to intellectual property theft. Raising your cyber hygiene is never really a bad thing.” 

The temptation for most organizations is to treat CMMC the way many treated every compliance framework that came before it: as a box to check, a certificate to post on their website, and another three-year cycle to survive. The organizations adapting are not simply generating documents more quickly, they’re using automation and AI to build tighter connections between compliance requirements, operational visibility, and security outcomes, with compliance teams involved from the start of every project rather than introduced at the end as an approval layer. 

US adversaries have already learned that periodic compliance is a vulnerability, and they’ve exploited it. Organizations must acknowledge that and build their compliance programs accordingly.