跳到正文
RCreddit.com·
暂不在当前实时榜单

One more 'you should try ExllamaV3/exl3 for flash next' appreciation post

AI 摘要

A user shared their positive experience with ExllamaV3/exl3 3bpw for running "flash next," noting impressive results. On a setup with 3x3090s and 128GB DDR4, they achieved 1500 prefill and 80 tps decode. A single 5090 with 128G DDR4 yielded 1500 prefill and 29 tps decode. Both configurations operated at 262k context with vision/spec decoding, leading the user to replace vllm/llama.cpp with ExllamaV3 for this model. They also plan to test the 4bpw version.

时间与来源

时间显示为 UTC

显示时区:UTC

本地时区尚不可用,暂时显示 UTC。

收录当时偏移:UTC+02026年9月21日 03:01 UTC

收录
2026年9月21日 03:01
来源类型
开发者社区

本站未收录正文。

前往源站阅读 →
来源·reddit.com