RCreddit.com
17
·开发者社区 · RSS
I wrote a GGUF inferencer from scratch, AMA
这条记录涉及编程工具或代码能力更新，适合开发者评估工作流变化和可复用价值。
一位开发者用 R 语言从零开始编写了一个 GGUF 推理器，专门针对特定模型。尽管这个推理器效率极低，每秒只能处理一个 token，但其主要目的是为了学习模型架构的工作原理，因为现有的指南已过时。…
*for a specific model, in the R language. veeery impractical.
60 seconds per token, yum 😋 not vibecoded, I just wanted to learn how the architecture works on our end lmao (and all of karpathy's guides are outdated 😢) writeup here , though it's kinda old news if you're already familiar with the math
I'd really like to make a useful inferencer next (you know, with an actual GPU kernel and memory management, in a language people actually use) but I can't find a good niche for one... llamacpp already runs on everything :/
主题标签Llama模型发布
原始关键词#inferencer#scratch#wrote#gguf#ama