Were designing a tiny autonomous research agent
开发者们正在设计一个微型自主研究代理，其基础模型仅有4300万个参数，并已在300万篇arXiv摘要上进行训练。该项目的目标是创建一个能够自主进行研究的本地代理，通过搜索网络并记录其发现来探索可能性。他们正在测试一个仅用科学、物理和技术内容训练的语言模型是否能做出新发现。开发团队正在积极寻求微调数据集的贡献，以及任何有助于塑造代理行为和实现其目标的建议。
This base model is only 43m parameters trained on 3m arXiv abstracts. We plan to continue pre-training and post training. If you create fine-tuning datasets or if you know of any datasets that can help shape the behavior for our goal we appreciate all contributors. The goal is to make a local agent that can autonomously do research. Its just a simple loop to search the web & document its findings as an experiment to see what is possible. If we train a language model on nothing but science, physics and technology can it make new discoveries?
We are testing this by creating fine-tuning examples that contain a pattern of asking questions and answering them until coming to a conclusion from first principles. If you have any suggestions to achieve the goal we are all ears. Please leave a comment.