RCreddit.com
15
·开发者社区 · RSS
If your GPU can run inference, it should be able to fine-tune too. [P]
这条记录涉及编程工具或代码能力更新，适合开发者评估工作流变化和可复用价值。
一位开发者创建了一种名为 USAF 的稀疏微调方法，旨在让能够运行 MoE 模型推理的 GPU 也能对其进行微调。该方法通过训练稀疏专家权重和路由器而非适配器来实现。作者在 AMD RX 6750 XT (12 GB) 上成功微调了 Qwen3-30B-A3B 模型。…
I spent the last few months building a new sparse fine-tuning method for MoE models called USAF.
The goal was simple: if your GPU can run inference on an MoE model, it should also be able to fine-tune it.
On my AMD RX 6750 XT (12 GB), I can fine-tune Qwen3-30B-A3B by training sparse expert weights and the router instead of adapters.
The project is completely open source under the Apache 2.0 license. I'm not trying to build a business, sell anything, or monetize it in any way—I just wanted to share something I built that I think is genuinely interesting.
主题标签模型发布开源代码
原始关键词#inference#should#able#fine#tune#gpu