RCreddit.com·
暂不在当前实时榜单
Update: Small model + Engram
A developer reported surprising coherence from a small 9b model after only 15 million tokens, attributing the success to borrowed embeddings and an LM_head down-projected from 5k to 2048 d_model. This method preserved approximately 65% of the data from a larger model's embedding, which significantly improved results compared to using pure 1 hot cross entropy that would have produced gibberish.
时间与来源
时间显示为 UTC
显示时区:UTC
本地时区尚不可用,暂时显示 UTC。
收录当时偏移:UTC+02026年9月17日 15:00 UTC
- 收录
- 2026年9月17日 15:00
- 来源类型
- 开发者社区
本站未收录正文。
前往源站阅读 →