Finally found a model my hardware can run at full precision: me

A developer, bored with optimizing local models, created a "tps counter for fingers" using real tokenizers. They achieved a personal best of around 2 t/s, which reportedly surpasses a 70B model on a laptop CPU and is approximately 76x slower than an 8B model running on a 4090. The developer also invited others to share their results.

Unlike typical model performance benchmarks, this report offers a human-centric comparison, pitting a developer's manual token generation speed against various AI models and hardware configurations.