Humaneval benchmark for Deepseek V4 Flash 0731 vs GLM5.3 Flash on 2x DGX Spark setup
Heat trend
Collecting trend data
The percentage is based on available heat signal, not comment count or independent people.
A user benchmarked Deepseek V4 Flash 0731 against GLM5.3 Flash on a 2x DGX Spark setup, focusing on HumanEval Pass@1 and HumanEval+ scores. GLM-5.3-Flash NVFP4 with 'Thinking Enabled' achieved 97.0% and 92.1% respectively, outperforming DeepSeek-V4-Flash-0731's 94.5% and 88.4%. While GLM-5.3-Flash showed better accuracy, DeepSeek-V4-Flash-0731 offered faster local stream speeds at ~70 tok/s compared to GLM's ~50 tok/s, with GLM's 256 context being a notable trade-off.
I have a 2 DGX Spark setup recently and I have been happily running Deepseek V4 Flash 0731. Since the release of GLM5.3 Flash and Qwen 3.8 Flash Next this week, a lot of folks are still waiting to see what model to run given their own hardware situations. I am very interested in running a GLM model locally and looks like nvfp4 would be a good option for my setup, but I have been hearing a lot of conflicting opinions (mostly negative) about GLM5.3 Flash on nvfp4 quant which gave me pause. So I decided to do a simple benchmark myself and hopefully this is useful for folks with the same setup:
Deepseek V4 Flash 0731 recipe: official checkpoint, fp8 kv, 1M context, 4 concurrent streams
GLM 5.3 Flash recipe: NVFP4 quant & Dflash2 drafter, fp8_e4m3 kv, 256k context, 6C
Model Thinking Mode HumanEval Pass@1 (Base) HumanEval+ (Adversarial Edge Cases) Total Benchmark Run Time Local Stream Speed GLM-5.3-Flash NVFP4 Thinking Enabled (high) 97.0% (159 / 164) 92.1% (151 / 164) 20m 52s ~50 tok/s (DFlash2) DeepSeek-V4-Flash-0731 Thinking Enabled (high) 94.5% (155 / 164) 88.4% (145 / 164) 38m 16s ~70 tok/s (MTP-5) GLM-5.3-Flash NVFP4 Direct Zero-Shot (off) 93.3% (153 / 164) 89.6% (147 / 164) 23m 32s ~50 tok/s (DFlash2) DeepSeek-V4-Flash-0731 Direct Zero-Shot (off) 92.7% (152 / 164) 87.8% (144 / 164) 14m 52s ~70 tok/s (MTP-5) So raw numbers tell you GLM 5.3f is a decent upgrade over DSv4f 0731 especially with thinking enabled. Unsloth saying their Q4 quant has around 92% accuracy but looks like nvfp4 still holds up pretty well (97% would have been a SOTA score not that long ago and this is not even max thinking). The major trade off is the 256 context. I am pretty sure 512GB+ VRAM (or 4x sparks) people will be able to run the fp8 model + 1M context without issues and I am jealous 🥹 Regardless, your own experience matters more than any benchmark out there.