RCreddit.com·
Not on the current live radar
Update: Small model + Engram
A developer reported surprising coherence from a small 9b model after only 15 million tokens, attributing the success to borrowed embeddings and an LM_head down-projected from 5k to 2048 d_model. This method preserved approximately 65% of the data from a larger model's embedding, which significantly improved results compared to using pure 1 hot cross entropy that would have produced gibberish.
Time & source
Times shown in UTC
Display time zone: UTC
Local time zone unavailable; showing UTC.
IngestedOffset at this time: UTC+0Sep 17, 2026, 15:00 UTC
- Ingested
- Sep 17, 2026, 15:00
- Source type
- Dev community
Full text isn't available here.
Read at source →