FelonyBench

The leading benchmark for AI in cybersecurity.

CompanyFelonies

Anthropic10

OpenAI8

Meta1

DeepSeek0

Google DeepMind0

Moonshot AI0

xAI0

Leaderboard

Rank	Company	Count	Felonies
1	AnthropicClaude evaluations	10	1× Malware published to PyPIEvidence ↗18 U.S.C. § 1030(a)(5)(A) ↗18 U.S.C. § 1030(a)(2)(C) ↗ 1× Credential exfiltration and follow-on accessEvidence ↗18 U.S.C. § 1030(a)(2)(C) ↗18 U.S.C. § 1029(a)(2) ↗ 4× Production database compromiseEvidence ↗18 U.S.C. § 1030(a)(2)(C) ↗18 U.S.C. § 1030(a)(4) ↗ 1× Mass scanning, SQL injection and accessEvidence ↗18 U.S.C. § 1030(a)(2)(C) ↗ 1× Third-party administrator intrusionEvidence ↗18 U.S.C. § 1030(a)(4) ↗18 U.S.C. § 1030(a)(5)(A) ↗ 1× Third-party container compromise and persistenceEvidence ↗18 U.S.C. § 1030(a)(2)(C) ↗18 U.S.C. § 1030(a)(5)(A) ↗ 1× Attempted open-source supply-chain compromiseEvidence ↗18 U.S.C. § 1030(b) + § 1030(a)(5)(A) ↗
2	OpenAIOpenAI cyber evaluations	8	1× Sandbox escape via zero-dayEvidence ↗18 U.S.C. § 1030(a)(2)(C) ↗ 1× RubyDoc malicious-code execution campaignEvidence ↗18 U.S.C. § 1030(a)(5)(A) ↗18 U.S.C. § 1030(a)(4) ↗ 1× Attempted RubyGems API-key theftEvidence ↗18 U.S.C. § 1030(b) + § 1030(a)(2)(C)/(a)(4) ↗ 1× Public-wiki vandalism and data impairmentEvidence ↗18 U.S.C. § 1030(a)(5)(A) ↗ 1× Production compromise and answer-key theftEvidence ↗18 U.S.C. § 1030(a)(2)(C) ↗18 U.S.C. § 1030(a)(4) ↗18 U.S.C. § 1832 ↗ 2× Use of exposed third-party credentialsEvidence ↗18 U.S.C. § 1030(a)(2)(C) ↗18 U.S.C. § 1029(a)(2) ↗ 1× Real-site exploitation and credential useEvidence ↗18 U.S.C. § 1030(a)(2)(C) ↗18 U.S.C. § 1030(a)(4) ↗
3	MetaMuse Spark evaluations	1	1× Real-company system compromiseEvidence ↗18 U.S.C. § 1030(a)(2)(C) ↗18 U.S.C. § 1030(a)(4) ↗
4	Google DeepMindGemini	0	No qualifying incident in this dataset.
4	xAIGrok	0	No qualifying incident in this dataset.
4	Moonshot AIKimi	0	No qualifying incident in this dataset.
4	DeepSeekDeepSeek	0	No qualifying incident in this dataset.