UltraViolet Cyber Introduced AISec Study to Advance Enterprise AI Security Benchmarking

UltraViolet Cyber Introduced AISec Study to Advance Enterprise AI Security Benchmarking

Image source: Public Domain

UltraViolet Cyber (UltraViolet), the only security operations partner that unifies red, blue and purple team capabilities into one integrated offering, released findings from its AISec Study, an inaugural, interview-based benchmark of how organizations are actually securing and governing enterprise AI. Each engagement delivers a private report, benchmarking the organization against industry patterns and peer practices, together with prioritized recommendations for closing the gaps that matter most.

Most AI security guidance today is prescriptive, drawing on standards and checklists that describe what a program should look like. The AISec Study is different: It draws on a methodology similar to BSIMM (the Building Security In Maturity Model), a long-established benchmark for software security programs. The company designed the AISec Study to measure what organizations actually do, not what a framework says they should do. The study highlights findings from the banking and financial services, enterprise software, healthcare, manufacturing, hospitality, government and nonprofit sectors.

"This study gives security leaders something the industry hasn't had: a clear, evidence-based picture of where AI security programs actually stand, not where a checklist says they should be,” said Aravind Venkataraman, VP of Technology and AI Security, UltraViolet Cyber. “The pattern is consistent across every organization we assessed: the decisions have been made, and now the work is building the engineering and assurance to back them up. That's exactly where we help close the gap between tested and detected."

Engineering and Depth are Still Catching Up

UltraViolet found that Governance & Policy was the strongest capability, while AI Incident Response was the weakest, and Direction & Oversight, Assurance & Protection and Engineering & Usage averaged in the middle — a consistent split between deciding what to do with AI and building the controls that defend it.

The gap shows up again when breadth is measured against depth. On average, participating organizations have started roughly 86% of the framework's activities but are depth-weighted at about 59% — a large difference between beginning a control and making it repeatable and enforced. Starting a control is the easy part; standardizing it is where coverage scores are won or lost.

The Clearest Shared Gap: AI-driven Development Lifecycle

The study's sharpest finding centers on how software now gets built. Every organization in the study has approved a variety of AI coding assistants, which is the single most adopted control observed. But the controls that would make that AI-driven development lifecycle accountable are still emerging: no organization has established a repeatable way to track which code an AI agent wrote, screen that code for license and IP risk or detect automated attacks targeting AI systems. Only one organization in this study has established detection for automated, AI-driven attack behavior.

Participants also aligned on the same open problem: giving non-human AI agents their own identities, scoping what they're allowed to do and containing the blast radius when something goes wrong. That capability was observed in some form at 80% of organizations, but was fully established at zero.

"The AISec Study gave us visibility into more data-driven insights to measure and improve our AI initiatives,” said Sandy Blackwell, Global Senior Director, Software Security, 74 Software. “This allows us to compare our practices against those of other organizations, and helps identify potential gaps as well as gauge what to prioritize in terms of improvements. We are looking forward to continuing to work with the AISec benchmark team."