OpenComputer | An Open Source Computer Built For Agents.
这条记录涉及编程工具或代码能力更新，适合开发者评估工作流变化和可复用价值。
Open Computer 是一个开源项目，旨在为非技术用户提供一个安全且用户友好的代理（agent）操作界面。它在一个隔离的虚拟机中运行，模拟真实的计算机环境，让代理能够像人类一样操作电脑，安装应用和操控用户界面。与传统的代理容器不同，Open Computer 专注于提供一个可视化、可理解的交互体验，避免了用户面对难以理解的命令行输出。…
Open Computer running in an isolated VM with inference running M4 Pro via LM Studio Gemma 4 13B QAT
Tim from AnythingLLM , where we have been building productive an on-device agent and AI assistant experience for the past 2.5 years now. I want to talk about a new experiment we are working on around agent UX for non-technical people.
Its clear that agent harnesses are only as powerful as the permissions you give them. For them to maximally useful the agent needs to basically own the entire PC so it can install apps and manipulate the UI when CLI or API calls fail.
This is clearly not a safe way to use agents and we all know it. We have seen a slurry of "agent containers" coming out like Apple's Containers , Microsoft MXC , and even Docker Sandboxes . Each of these simply wrap the agent in a micro-vm - which is a step in the right direction.
However, the issue I take with this is the experience that agent show to users. When the agent is blindly executing commands no mere mortal could comprehend there is basically nothing for a user to peek into or observe while this terminal like output pours into the console. Hopefully the end result was worth the tokens.
We wanted to see if there was a way to surface a regular computer interface to the user. Basically an agent container that looks, feels, and operates like a real computer for a human, but then outfit for and agent harness to go nuts on.
So that is what Open Computer is - a computer that is manageable for a human and useful for agents. We did however want to bring this idea of a Perplexity Computer to local AI - since it seems almost all harnesses are requiring large models with 256K contexts to even be useful.
The above video is Gemma 4 13B QAT @ 32K context and it works! It can manipulate the browser (for more than 50% less tokens than raw browser-use!), leverage native app accessibility trees for entering data in native apps, and has a bunch of tools pre-configured specifically for small context windows. Designing this way means hooking this up to a big cloud model saves you money as we aggressively prune context to prevent context bloat.
The "base" image each agent computer inherits is about 3GB. Each agent itself is only about 100MB of space, less than than in RAM, and the disk is aggressively compacted to keep it small - no matter how much stuff it installs to do the task you give it.
The computer itself is quite lightweight: - Debian 13.5 - XFCE4 " Riced " to look like windows 10. - Pi.dev as the main harness - With Hermes memory and a management UI built in isolated to the agent computer.
The inference for this is 100% agnostic to the VM's. So you can run inference for one agent using your local compute, another using cloud or a local server, and so on. All of which have their own isolated computer that is totally virtualized so it cannot harm your underlying host.
Another reason we chose to do this, was that the paradigm of computer use is functionally broken as a concept. An LLM to highjack your keyboard and mouse and struggle to click a button relying exclusively on screenshots is not only frustrating to watch, but a huge token sink. Open Computer does not screenshot anything for navigation.
Lastly, we wanted to put the human at the center of this. So there is a whole UI/UX about the agent pinging the user and the user be able to act in the computer in a way that makes sense - like logging in to a website, solving a captcha, or whatever else puzzles the agent can now be solved collaboratively by the human and the agent.
This is all being built in open source inside AnythingLLM https://github.com/Mintplex-Labs/anything-llm/blob/master/open-computer/README.md - if this project interests you a star goes a long way.
We ideally want to build out a UI inside our app so users can easily spin up/down these computers for agents and power it with their local hardware - including NPU.
Open Computer though is open to anyone and can be implemented anywhere - even as an MCP even by the host agent harness you are using. Its agents all the way down, haha.
Anyway, I would like to know what people think about this concept. I have really enjoyed "seeing" the work agents do and click around on and read. It feels more like coworking than just some black box executing that hopefully gives me what I want.
this is all still very early, so there are for sure bugs and things to improve!