Back
RCreddit.com
26
·5 hr ago·Dev community · RSS

I made Claude's web research 18× cheaper with 2 lines of setup

View original
AI summary

A new tool, Webify, significantly reduces token usage for Claude's web research by extracting only relevant sections from webpages.…

I've been using Claude Code daily, and one thing kept bothering me.

Whenever Claude calls WebFetch, it often dumps 3,000–15,000 tokens from an entire web page into the context window, even if the answer is buried in a single section.

Multiply that across a few documentation pages, and you're spending thousands of unnecessary tokens just to answer one question.

So I built Webify.

Instead of sending the whole page to Claude, Webify parses the HTML into a DOM graph, identifies the parts relevant to your query using BM25 and a BFS traversal, and returns only the relevant subtree.

In practice, Claude usually gets 80–300 tokens instead of several thousand.

I ran a blind benchmark on 15 unseen queries (Sonnet as the judge):

- Webify: 68/75 (91%)

- Deep Research: 73/75 (97%)

The gap wasn't accuracy, it was completeness. Deep Research simply reads more pages. For most developer workflows, the answers were effectively the same while using a fraction of the tokens.

The pipeline is pretty simple:

- Parse HTML into a DOM hierarchy

- Score nodes using BM25

- Traverse nearby nodes with BFS to preserve context

- For search, build graphs from multiple pages in parallel and synthesize the results

No embeddings. No vector database. Just retrieving the part of the page that's actually relevant instead of making Claude read everything.

Installation takes about 30 seconds:

pip install webify-mcp claude mcp add webify -- webify-mcp

Github Link: https://github.com/kunal12203/webify-mcp/

No config files. It works with Claude Code, Cursor, Windsurf, VS Code, Zed, or anything that supports MCP.

It's totally open source under an MIT license, and I'd love to hear where it breaks or how it can be improved; PRs are welcome.

TopicsClaudePlans & limits
Keywords#research#cheaper#lines#setup#made
View originalreddit.com
Single source, no cross-check yet