GLM 5.2 FP8 with FP8 KV - Terminal-Bench 2.1 = 79.8 (with one time-out that I didnt re-run)
该测试旨在比较官方结果与使用 FP8 和 FP8 KV 的模型性能，测试环境为 H200 上的 SGLang 设置。在 TERMINAL-BENCH 2.1 的最终结果中，模型完成了 89 项任务，通过了 71 项（79.8%），失败了 17 项，并有 1 项因超时而报错。…
I wanted to test the official results vs fp8 + fp8 kv. basic sglang setup on H200.
If anyone wants one of the official tests do ping me.
I didnt rerun the one so it might go up a bit :)
TERMINAL-BENCH 2.1 — FINAL RESULTS (mymodel via mini-swe-agent) TOTAL: 89 tasks PASSED: 71 (79.8%) FAILED: 17 ERRORED: 1 Input tokens 218,656,815 Cache tokens 216,036,672 Output tokens 4,659,650 Cache hit rate 98.8% New input tokens 2,620,143 FAILED (17) — completed but wrong answer configure-git-webserver db-wal-recovery dna-assembly dna-insert extract-moves-from-video filter-js-from-html gcode-to-text install-windows-3.11 model-extraction-relu-logits mteb-leaderboard protein-assembly query-optimize raman-fitting regex-chess torch-pipeline-parallelism video-processing winning-avg-corewars ERRORED (1) — timed out torch-tensor-parallelism [VerifierTimeoutError — not rerun]