M5 Max users: what models are you using & what tk/s are you getting?

模型发布

时间与来源

发布: 09/06 01:32
收录: 09/06 16:00
来源类型: 开发者社区
档位: 社区
信源状态: 正常

I was using antirez’s ds4 for a while and getting around 20 tk/s, which worked for my purposes. But I know there have been big advancements between Qwen, the DS4 vision model, and GLM.

I’m not sure how the quants affect performance, so what’s the best thing to run right now & how fast is it?