Cybersecurity leader CrowdStrike has teamed up with Meta to release CyberSOCEval, a new set of open-source benchmarks that measure how well artificial intelligence (AI) systems perform in real-world cybersecurity operations.
The tool combines Meta’s CyberSecEval framework with CrowdStrike’s threat intelligence and cybersecurity expertise, and is designed to help security teams and developers test, compare, and improve large language models (LLMs) for use in security operations centers (SOCs).
“When two leaders like CrowdStrike and Meta come together, it’s larger than collaboration, it’s about setting the direction of cybersecurity for the AI era,” said Daniel Bernard, chief business officer at CrowdStrike. “By combining CrowdStrike’s adversary intelligence and leadership in AI-native cybersecurity, with Meta’s AI research expertise and vast dataset, we’re helping customers — and cybersecurity as a sector — adopt AI systems with confidence.
“This partnership sets a new bar for how AI in the SOC should be built and deployed, empowering defenders to stay ahead of the adversary,” Bernard added.
Today’s defenders face an avalanche of security alerts and increasingly sophisticated attacks. While many teams are beginning to use AI, especially LLMs, they often lack clear ways to evaluate which tools truly improve speed, accuracy, and resilience, according to CrowdStrike.
CyberSOCEval aims to fill that gap by providing practical tests across key workflows like malware analysis and threat investigation.
By simulating real-world attack scenarios and expert-designed challenges, the benchmarks will allow organizations to see how well AI systems hold up under pressure. Security teams then may identify where AI delivers the most value, while developers gain guidance on how to improve models for maximum impact.
Vincent Gonguet, director of product, GenAI at Meta, said the collaboration highlights Meta’s commitment to open-source AI, especially as large language models become powerful tools for organizations of all sizes.
“With these benchmarks in place, and open for the security and AI community to further improve, we can more quickly work as an industry to unlock the potential of AI in protecting against advanced attacks, including AI-based threats,” said Gonguet.
CyberSOCEval is now available for the AI and security community through Meta’s CyberSecEval framework.