Qwen 3.8 - 27B is a game changer

So a bit of context, I am a cybersecurity senior analyst I am interested in LLMs for that field especially with MCPs to connect them to the tools or for writing scripts

I started this field by doing assembly language reading for hacking games when I was a teenager then that became malware analysis then I started to analyze traffic and logs at work for a living Before work I competed in cybersecurity competitions known as capture the flag to solve only one category of the challenges and that is malware analysis

Now here is the scene in LLM x Cybersecurity

Entry-level CTF challenges (I used to solve around 2017-2018, got first job in 2019) were solved and saturated by LLMs a long time ago (See intercode CTF benchmark)

Then High level CTFs (NYU CTF Bench, CSAW challenges, and CyBench) these were solved a while ago

Today we have CyberGym (vulnerability description (CVE report not real details) plus code base find vulnerability That was solved

Then ExploitGym (the one recent OpenAI model escaped and hacked hugging face to find solution, the 2 vulnerabilities themselves are trivial (JWT spoofing with no server side checks for Jfrog sandbox and JS + Python template in open source data viewer of Hugging face) Still impressive for a fully autonomous model

ExploitGym gives vulnerability details + exploit technique/details and codebase (I wonder how/why OpenAI model escaped it in the first place those details should manage it this is why I feel it is staged but whatever right)

ExploitBench More realistic threat vector 1-day not 0-day (0-day is a novel vulnerability no one else knows about it hence the name, 1-day is a known vulnerability patched, if in an open source project you can compare patches before and after and know it but not as valuable as 0-day since people will/should update their software, 0-days can and are used against high profile targets like in some 3rd world countries used for surveillance (and possibly non-3rd world countries ?!) but you usually can’t use a 1-day for that but still)

So ExploitBench is 1-days in V8 engine (used in chrome, electron vs code almost everything!!)

Gives the model a patch diff and codebase base that is it

Models are going wild on ExploitBench and ExploitGym but haven’t gotten there yet at the moment and only recent frontier models improved massively instead of near zero across the board

This is a massive risk indicator Imagine models capable of exploiting any and every piece of software written by less capable people or other weaker LLMs (complex vulnerabilities!)

Not only that but global elite CTFs like DEF CON

You have an OpenSage harness (and ADK like Google adk or Anthropic adk, however this OpenSage ADK asks the model to design its own agent/s!, with whatever tools or sandbox environments it finds fitting for the Subagent to do the assigned subtask effectively, it can even design its own MCPs!, models don’t fully utilize that yet but some do and improved form like 39% to ~60% just by using that harness (same model!) )

- Now my own benchmarks are simple

Some malware samples that I learned during my career and consider personal milestones

And one SIEM analysis challenge (logs that contains an attack plus normal behavior)

I tests on lots of models (Qwen3.6, heavily trimmed down Minimax 2.5 …etc, on my Strix halo and compared with opus at the time (the best opus at the time was 4.5!, it was better solved major stuff but didn’t implement the code required to decompose the malware and analyze its second stage)

Now enter Qwen 3.8 27b
 3.
 What was SOTA like 6 months ago or around beginning of this year
 Now we have better local models at the same task

Qwen 3.8 devoured the malware that Opus couldn’t

I had some time analyzing and remembering its custom implementation (it has custom RC4 decryption routine, that is tricky with string manipulation! And Strix halo isn’t the fastest LLM machine

By the time I remembered that malware and how it works

Qwen had already extracted and disassembled the decrypted payload!