AIKIDO ARENA
AI model benchmarks for cybersecurity
See the best models for cybersecurity. We run frontier models against vulnerabilities in real-world software. Updated as new models ship.
Last updated: 2026-08-21
Changelog
October 28, 2026
- Retired GPT-5.6 Terra, Grok 4.6, DeepSeek V4 Pro 0813, DeepSeek V4 Flash 0731, GLM-5.3 Flash, and Claude Haiku 5.5
October 27, 2026
- Added DeepSeek V4.1 Flash 1017 results, which replace the 0910 snapshot
- It found 28/32 vulnerabilities pass@3, tying DeepSeek V4 Pro 0813 and one behind GPT-6 Astra.
- It scored 86.4% precision, up from 78.9% for the 0910 snapshot and the highest of any DeepSeek model on the board.
- One run found 70.8% on average; combining three got it to 87.5%.
- At $2.40 per CVE found, it ties Claude Haiku 5.5 and trails only GPT-6 Luna ($2.01).
- Added MiniMax M3.1 Flash results
- It found 24/32 vulnerabilities pass@3, compared with 28/32 for DeepSeek V4.1 Flash 1017.
- One run found 59.4% on average, against 70.8% for DeepSeek V4.1 Flash 1017; combining three got it to 75%.
- It scored 80.5% precision, which means a false-positive rate about 40% higher than DeepSeek V4.1 Flash 1017's.
- At $4.20 per CVE found, it costs well above DeepSeek V4.1 Flash 1017 ($2.40).
- It found a deserialization bug in spring-kafka in all three runs, which Opus 5 and Qwen3.8 Max missed every time.
- In the traces, it compares permissions across neighboring endpoints and checks what access an attacker would already need, which keeps noise out of its reports.
- It can also get stuck in reasoning loops; one investigation produced over 1M characters of reasoning.
- It holds up reasonably well, but against today's alternatives we wouldn't recommend it for cybersecurity tasks.
- Retired DeepSeek V4.1 Flash (0910)
October 13, 2026
- Added Muse Wonder 1.0 results
- It found 26/32 vulnerabilities pass@3, the same count as Opus 5 and Grok 4.6.
- One run found 57.3% on average; combining three got it to 81.3%. Only 12 CVEs turned up in all three runs, so it does best when you run it three times.
- It scored 80.8% precision, ahead of only the DeepSeek models on the board. Weaker reports assumed an attacker could reach the bug, or that downstream code would misuse it, without checking either.
- It drops theories that don't hold up. In one trace it suspected a missing ownership check, followed every caller, found each one guarded, and backed off after 27 turns.
- At $71.75 per CVE found, it costs more than Opus 5 ($64.96). Only 41.3% of its input came from cache, compared with 79.5% to 91.1% for the GPT and Haiku models we checked, so far more tokens get billed at full price.
October 8, 2026
- Added Claude Haiku 5.5 results
- It found 17/32 vulnerabilities pass@3, tying GPT-6 Luna. Every CVE it found, Opus 5 found too.
- It scored 94% precision, higher than Opus 5's 87%.
- One run found 36.5% on average; combining three got it to 53.1%. Only 6 CVEs turned up in all three runs, the fewest of any model on the leaderboard, so extra runs pay off more for Haiku than for anything else we've tested.
- At $2.40 per CVE found, it's the third cheapest model on the board, behind GPT-6 Luna and DeepSeek V4.1 Flash. Opus 5 costs $64.96 per CVE found.
October 2, 2026
- Added Xiaomi MiMo 2.6 Pro results
- It found 25/32 vulnerabilities pass@3, matching GPT-6.1 Sol, Kimi K3 and GLM 5.3 and just one behind Opus 5
- It scored 88% precision, ahead of all Qwen and DeepSeek models
- One run found 64.6% on average; combining three got it to 78.1%
- All that for just $11.79 per CVE found
September 29, 2026
- Added Grok 4.7 results
- 4.7 found 3 vulnerabilities that 4.6 missed every time, while 4.6 found 2 that 4.7 missed. The trade-off is repeatability. 4.7 found 18/32 in all three runs, versus 21/32 for 4.6. More reach for a little less consistency.
- Added GPT-6 Luna and GPT-6 Sol results
- We ran GPT-6 Luna and Sol on our 32-CVE cyber benchmark and the results were unexpected! Luna rediscovered 53.1%. Sol reached 68.8%.
- Neither beat GPT-5.6 variants on recall. But both got A LOT cheaper per vulnerability found: Luna $3.43 → $2.01/CVE and Sol $56.88 → $34.18/CVE.
September 11, 2026
- Added GPT-6 Astra and GLM-5.3 Flash results.
A strong model is just the beginning.
Aikido's harness turns reasoning into high-quality security findings across your application, whether it's Code Security Audit reading your source or Aikido Attack testing your running app.
Contributors
Use keyboard
to navigate through articles
.svg)
Get secure today,
quickly and for free.
Secure your code, cloud, and runtime in one central system. Connect a repo to discover what the reasoning agents find in your codebase.
No credit card required | Scan results in 32 seconds.
Trusted by 150k+ orgs
