I made my own cybersecurity benchmark and ran Qwen3.8 27B, here's how a local model actually does at hacking

A user developed a cybersecurity benchmark and tested Qwen3.8 27B, a local AI model, on its hacking capabilities. The model was run on a llama.cpp RPC pool across a 3090 and a 3080 in two Proxmox nodes, connected via a 2.5G link, providing sufficient concurrent transactions per second for multiple agents. The user is seeking feedback on the methodology and task mix of this initial benchmark.

This is the first publicly shared cybersecurity benchmark for a local AI model, unlike previous benchmarks that focused on cloud-based or proprietary systems.