Keyboard Shortcuts

Focus search/
Go to Directoryg d
Go to Trendingg t
Go to Exploreg e
Go to Homeg h
Close modal / blur searchEsc
Show this help?
Press ? anytime to toggle this panel

Best Web Scraping Agents

Automated web scraping, data extraction, and browser automation agents.

55 agents found · Sorted by trust score

AgentDescriptionCapabilitiesProtocolsStarsType

Lightpanda

Tool by lightpanda-io
91.0

The headless browser designed for AI agents and web automation

web-browsingscrapingautomationheadless-browser
RESTCDP
★ 22.4k Zig
91.0
$ pip install lightpanda click to copy

Scrapling

Mcp Server by D4Vinci
72.9

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

api-integrationdata-extractionweb-browsingweb-scraping
MCP
★ 28.9k Python
72.9

Chrome Devtools Mcp

Mcp Server by ChromeDevTools
72.5

Chrome DevTools for coding agents

browser-automationweb-browsing
MCP
★ 30.0k TypeScript
72.5

Browser Tools Mcp

Mcp Server by AgentDeskAI
67.8

Monitor browser logs directly from Cursor and other MCP compatible IDEs.

anthropicbrowser-automationlanguage-modelslogging
MCP
★ 7.1k JavaScript
67.8

Firecrawl Mcp Server

Mcp Server by firecrawl
67.5

🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.

api-integrationci-cdinformation-retrievallanguage-models
MCP
★ 5.8k JavaScript
67.5

Brightdata Mcp

Mcp Server by brightdata
64.4

A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.

api-integrationbrowser-automationlanguage-modelsweb-browsing
MCP
★ 2.2k JavaScript
64.4

spider

Agent by spider-rs
64.0

Web crawler and scraper for Rust

automationweb-scraping
REST
★ 2.3k Rust
64.0

Awesome Hacking Lists

Mcp Server by taielab
63.0

A curated collection of top-tier penetration testing tools and productivity utilities across multiple domains. Join us to explore, contribute, and enhance your hacking toolkit!

language-modelstestingweb-browsingweb-scraping
MCP
★ 1.3k
63.0

Web Eval Agent

Mcp Server by refreshdotdev
61.9

An MCP server that autonomously evaluates web applications.

testingweb-browsingweb-scraping
MCP
★ 1.2k Python
61.9

Browser Use Mcp Server

Mcp Server by kontext-dev
61.0

Browse the web, directly from Cursor etc.

browser-automationweb-browsingweb-scraping
MCP
★ 812 Python
61.0

Vibetest Use

Mcp Server by browser-use
60.5

Vibetest MCP - automated QA testing using Browser-Use agents

browser-automationtestingweb-browsing
MCP
★ 777 Python
60.5

Web Agent Protocol

Mcp Server by OTA-Tech-AI
59.6

🌐Web Agent Protocol (WAP) - Record and replay user interactions in the browser with MCP support

browser-automationlanguage-modelsweb-browsingweb-scraping
MCP
★ 495 Python
59.6

Pointer

Mcp Server by etsd-tech
59.4

MCP tool: let you point at DOM elements for your favorite agentic coding tool. Let AI see what you see.

browser-automationcode-generationweb-browsing
MCP
★ 568 TypeScript
59.4

Stealth Browser Mcp

Mcp Server by vibheksoni
58.9

The only browser automation that bypasses anti-bot systems. AI writes network hooks, clones UIs pixel-perfect via simple chat.

anthropicapi-integrationbrowser-automationlanguage-models
MCP
★ 369 Python
58.9

Webresearch

Mcp Server by mzxrai
58.1

MCP web research server (give Claude real-time info from the web)

anthropicinformation-retrievallanguage-modelssearch
MCP
★ 292 JavaScript
58.1

Web3 Research Mcp

Mcp Server by aaronjmars
56.3

Deep Research for crypto - free & fully local

cryptocurrencyinformation-retrievalsearchweb-browsing
MCP
★ 152 TypeScript
56.3

Kindly Web Search Mcp Server

Mcp Server by Shelpuk-AI-Technology-Consulting
55.8

Kindly Web Search MCP Server: Web search + robust content retrieval for AI coding tools (Claude Code, Codex, Cursor, GitHub Copilot, Gemini, etc.) and AI agents (Claude Desktop, OpenClaw, etc.). Suppo

code-generationcode-managementgitgithub
MCP
★ 232 Python
55.8

Agentql Mcp

Mcp Server by tinyfish-io
55.7

Model Context Protocol server that integrates AgentQL's data extraction capabilities.

api-integrationlanguage-modelsweb-browsingweb-scraping
MCP
★ 150 JavaScript
55.7

Read Website Fast

Mcp Server by just-every
55.1

Quickly reads webpages and converts to markdown for fast, token efficient web scraping

api-integrationci-cdcode-generationdata-extraction
MCP
★ 137 JavaScript
55.1

Web Scout Mcp

Mcp Server by pinkpixel-dev
54.1

A powerful MCP server extension providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI a

api-integrationdata-extractioninformation-retrievalsearch
MCP
★ 125 JavaScript
54.1

Oxylabs Mcp

Mcp Server by oxylabs
54.0

Official Oxylabs MCP integration

api-integrationci-cdlanguage-models
MCP
★ 86 Python
54.0

Crawl4Ai Mcp Server

Mcp Server by sadiuysal
52.4

🕷️ A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents. Similar to Firecrawl's API but self-hosted and free. Perfect

api-integrationcode-generationlanguage-modelsopenai
MCP
★ 62 Python
52.4

Crawlbase Mcp

Mcp Server by crawlbase
52.2

Crawlbase MCP Server connects AI agents and LLMs with real-time web data. It powers Claude, Cursor, and Windsurf integrations with battle-tested web scraping, JavaScript rendering, and anti-bot protec

api-integrationlanguage-modelstestingweb-browsing
MCP
★ 52 JavaScript
52.2

Decipher Research Agent

Mcp Server by mtwn105
51.7

Turn topics, links, and files into AI-generated research notebooks — summarize, explore, and ask anything.

api-integrationci-cdfile-managementfilesystem
MCP
★ 145 TypeScript
51.7

Browser Agent

Mcp Server by imprvhub
51.3

A Model Context Protocol (MCP) integration that provides Claude Desktop with autonomous browser automation capabilities. This agent enables Claude to interact with web content, manipulate DOM elements

api-integrationbrowser-automationweb-browsingweb-scraping
MCP
★ 35 TypeScript
51.3

Civyk Winwright

Mcp Server by civyk-official
50.4

Playwright-style MCP server for Windows desktop, system, and browser automation. 59 tools for WPF, WinForms, Win32, Chrome/Edge via Model Context Protocol.

browser-automationci-cdtestingweb-browsing
MCP
★ 48 PowerShell
50.4

Sample Agentic Ai Web

Mcp Server by aws-samples
48.2

This project demonstrates how to use AWS Bedrock with Anthropic Claude and Amazon Nova models to create a web automation assistant with tool use, human-in-the-loop interaction, and vision capabilities

anthropicawscloudlanguage-models
MCP
★ 49 Python
48.2

Aoai Web Browsing

Mcp Server by kimtth
41.1

A minimal Model Context Protocol 🖥️ server/client🧑‍💻with Azure OpenAI and 🌐 web browser control via Playwright.

azurebrowser-automationcloudlanguage-models
MCP
★ 31 Python
41.1

nstbrowser-ai-agent

Tool by nstbrowser
40.0

Nstbrowser AI agent for browser automation with advanced fingerprinting

tool-calling
REST
TypeScript
40.0
$ npm install nstbrowser-ai-agent click to copy

@aws-sdk/util-user-agent-browser

Framework by aws-sdk-bot
40.0

[![NPM version](https://img.shields.io/npm/v/@aws-sdk/util-user-agent-browser/latest.svg)](https://www.npmjs.com/package/@aws-sdk/util-user-agent-browser) [![NPM downloads](https://img.shields.io/npm/

REST
TypeScript
40.0
$ npm install @aws-sdk\/util-user-agent-browser click to copy

agent-browser

Agent by vercel-release-bot
40.0

Headless browser automation CLI for AI agents

REST
TypeScript
40.0
$ npm install agent-browser click to copy

@mlc-ai/web-llm

Agent by akaashrp
40.0

Hardware accelerated language model chats on browsers

conversational
REST
TypeScript
40.0
$ npm install @mlc-ai\/web-llm click to copy

@jarvis-agent/web

Framework by lsustc
40.0

Browser/Web environment support for Jarvis Agent framework.

REST
TypeScript
40.0
$ npm install @jarvis-agent\/web click to copy

agentpage

Framework by xiaoshanwen
40.0

Embeddable AI Agent SDK for browsers — let AI operate your web pages via tool-calling

embeddingstool-calling
REST
TypeScript
40.0
$ npm install agentpage click to copy

@deway-ai/web-sdk

Framework by noy-getrise
40.0

Deway's Web SDK

REST
TypeScript
40.0
$ npm install @deway-ai\/web-sdk click to copy

@azure/ai-projects

Framework by microsoft1es
40.0

Azure AI Projects client library.

REST
TypeScript
40.0
$ npm install @azure\/ai-projects click to copy

@probelabs/probe-chat

Agent by GitHub Actions
40.0

CLI and web interface for Probe code search (formerly @probelabs/probe-web and @probelabs/probe-chat)

code-generationconversationalsearch
REST
TypeScript
40.0
$ npm install @probelabs\/probe-chat click to copy

slapify

Agent by vguleria
36.0

AI-powered browser automation — autonomous task agent, performance auditing, and E2E test flows in plain English

REST
TypeScript
36.0
$ npm install slapify click to copy

@kritchoff/agent-browser

Agent by kritchoff
36.0

Headless browser automation CLI for AI agents

REST
TypeScript
36.0
$ npm install @kritchoff\/agent-browser click to copy

@ujeebu-org/langchain

Agent by ujeebu
36.0

LangChain integration for Ujeebu Extract API

REST
TypeScript
36.0
$ npm install @ujeebu-org\/langchain click to copy

@docdigitizer/langchain

Agent by joaocostafernandes
36.0

LangChain document loader for the DocDigitizer document processing API

retrieval-augmented-generation
REST
TypeScript
36.0
$ npm install @docdigitizer\/langchain click to copy

firecrawl-aisdk

Framework by hello_sideguide
36.0

Firecrawl tools for Vercel AI SDK - Web scraping and search capabilities for AI applications

searchtool-calling
REST
TypeScript
36.0
$ npm install firecrawl-aisdk click to copy

@promptbook/anthropic-claude

Framework by hejny
36.0

Promptbook: Turn your company's scattered knowledge into AI ready books

conversationalembeddingsfunction-callingretrieval-augmented-generation
REST
TypeScript
36.0
$ npm install @promptbook\/anthropic-claude click to copy

@agent-infra/browser-context

Agent by GitHub Actions
30.0

get browser context for AI Agent

REST
TypeScript
30.0
$ npm install @agent-infra\/browser-context click to copy

bowser

Agent by GitHub Actions
30.0

Lightweight browser detector

retrieval-augmented-generation
REST
TypeScript
30.0
$ npm install bowser click to copy

trustcalljs

Tool by siddicky
30.0

Utilities for validated tool calling and extraction with retries using LLMs - TypeScript port of trustcall

tool-calling
REST
TypeScript
30.0
$ npm install trustcalljs click to copy

@vapi-ai/web

Agent by arvindrk94
24.0

This package lets you start Vapi calls directly in your webapp.

REST
TypeScript
24.0
$ npm install @vapi-ai\/web click to copy

@azure/ai-agents

Framework by microsoft1es
16.0

Azure AI Agents client library.

REST
TypeScript
16.0
$ npm install @azure\/ai-agents click to copy

@agentic/firecrawl

Framework by fisch0920
16.0

Agentic SDK for Firecrawl.

REST
TypeScript
16.0
$ npm install @agentic\/firecrawl click to copy

browser-use

Agent by Gregor Zunic
12.0

Make websites accessible for AI agents

web-browsing
REST
Python
12.0
$ pip install browser-use click to copy

Related Categories

🌐 Web & Scraping