I built a local-first hybrid router for AI Agent Skills (sub-20ms, zero tokens, runs on CPU) [P]
- 发布
- 09/06 16:34
- 收录
- 09/06 20:00
- 来源类型
- 开发者社区
- 档位
- 社区
- 信源状态
- 正常
If you use agentic workflows with custom skills or rules (Cursor rules, Claude Code slash commands, OpenCode, etc.), you have probably run into the routing trade-off:
- Stuff every skill definition into the system prompt (destroys your context window and degrades instruction-following).
- Use an LLM router turn to classify the user prompt (costs money, wastes 1,000+ tokens, and adds 2+ seconds of network latency).
To solve this, I built Routed ; an open-source, local-first hybrid router for agent skills that runs 100% offline on your CPU.
Routed indexes your installed skill directories and evaluates prompts through a 4-part hybrid scoring pipeline:
* Dense Vector Embeddings (60%): Runs quantized ONNX models (Arctic Embed S / MiniLM) locally on CPU. * Lexical BM25 (25%): Okapi BM25 for strict keyword relevance. * Exact / Alias Match (10%): Direct command and alias matching. * Metadata (5%): Recency and usage heuristics.
The entire lookup completes in under 20ms without sending a single byte of prompt data over the wire.
Cursor, Claude Code, LM Studio, Ollama, Antigravity IDE, Windsurf, OpenCode, Continue, Codex, and I just dropped support for MCP Servers!!
And although v1.0 dropped last night, I just shipped v1.1.0 with two major additions based on early feedback:
- Model Context Protocol (MCP) Server (routed mcp): Instead of loading 20+ tool schemas into your GPU's context window, your local model only sees a single route_skill tool. Routed executes on CPU, selects the exact skill needed, and injects only that schema on demand.
- Native Multilingual Understanding: The embedding pipeline now natively understands input across 100+ languages (German, Spanish, French, Japanese, etc.) and automatically decomposes compound nouns (like German Speicherleck ), mapping prompts directly to the correct skill without language tags or manual translation.
Inside your agent chat, you can just use /route to trigger the best skill(s) dynamically. It also handles compound intents (e.g., matching multiple skills when a prompt asks for two distinct tasks).
Pre-built binaries are available on GitHub Releases (macOS .pkg, Linux .deb, Windows .exe), or you can build it from source via Node.
Check it out and let me know what you think or if there are other environments you would like added!