I spent $200 benchmarking 9 cloud browsers against 400 bot-protected websites

I recently spent about $200 on cloud browser subscriptions to see how well they actually work against modern anti-bot systems.

If you're building AI agents or web scrapers, parsing the page usually isn't the hard part. Getting the browser onto the page without hitting a CAPTCHA or block is.

A lot of browser-agent benchmarks focus on how well an agent completes tasks once the page is accessible. We wanted to test something more basic: can it reliably get onto the site in the first place?

So we tested 9 cloud browser products and open-source frameworks across 400 websites protected by Cloudflare, DataDome, PerimeterX, Akamai and other anti-bot systems.

Full disclosure: we're building one of the products in the benchmark (bro), so we obviously have skin in the game. That's why we've published the methodology and raw results so the benchmark can be reproduced, challenged, or improved.

A few rules we used:

- No automated CAPTCHA solvers — only passive stealth capabilities.

- The open-source frameworks used the same proxy network to keep IP quality consistent.

- We allowed extra time after navigation before deciding whether a page loaded successfully.

- Around 4,000 screenshots were classified with an LLM and then manually checked.

Results:

- bro — 83.50%

- Browserbase — 79.75%

- Browser Use — 79.75%

- Browserless — 75.50%

- Firecrawl — 73.50%

- Hyperbrowser — 63.25%

- Obscura — 39.50%

- Selenium — 37.00%

- Playwright — 36.25%

bro ended up finishing first, which was obviously a nice result for us, but there were also some interesting cases where individual browsers performed very differently depending on the protection being used.

The full breakdown includes costs, individual site results, methodology, screenshots, and edge cases:

Evals breakdown: https://getbro.ws/blog/cloud-browsers-benchmark

Raw benchmark repo: https://github.com/jsonifyco/browser-benchmarks