Which model, which harness? I have data for you.

A developer has created a benchmark and leaderboard on airbench.ai to test various model and harness setups on real-world tasks such as basic math, vision, computer use, and coding. Initial findings indicate that MTP variants negatively impact performance, with nvfp4-mtp averaging 52% compared to 70% without it, and swift on pi dropping from 96% to 55% with MTP. A 65k context also reduces performance (45–61%), while Gemma-4-26b is fast but achieves a maximum of 47%.