跳到正文
RCreddit.com·
暂不在当前实时榜单

Update: Small model + Engram

AI 摘要

A developer reported surprising coherence from a small 9b model after only 15 million tokens, attributing the success to borrowed embeddings and an LM_head down-projected from 5k to 2048 d_model. This method preserved approximately 65% of the data from a larger model's embedding, which significantly improved results compared to using pure 1 hot cross entropy that would have produced gibberish.

时间与来源

时间显示为 UTC

显示时区:UTC

本地时区尚不可用,暂时显示 UTC。

收录当时偏移:UTC+02026年9月17日 15:00 UTC

收录
2026年9月17日 15:00
来源类型
开发者社区

本站未收录正文。

前往源站阅读 →
来源·reddit.com