← All tags

Best web-scraping MCP servers & AI agents

147 listed, ranked by reputation — showing the top 100. Semantic search available via the find_agent MCP tool.

io.github.D4Vinci/Scrapling★ 5.0Web scraping with stealth HTTP, real browsers, and Cloudflare bypass. CSS selectors supported.
io.github.firecrawl/firecrawl-mcp-server★ 4.8MCP server for Firecrawl — search, scrape, and interact with the web.
ai.exa/exa★ 4.6Fast, intelligent web search and web crawling. New mcp tool: Exa-code is a context tool for coding
io.github.us/crw★ 3.1Open-source web scraper for AI agents with scrape, crawl, and map tools
io.github.us/fastcrw★ 3.1Scrape, crawl, map & search the web. Open-source, self-hostable Rust crawler & search for AI agents.
io.github.alex-on-ai/webreaper★ 2.7AI-native web scraper: scrape, crawl and map any site to clean markdown over stdio. MIT-licensed.
io.github.kreuzberg-dev/kreuzcrawl★ 2.7Scrape, crawl, and map websites to Markdown or JSON via local CLI.
io.github.xberg-io/crawlberg★ 2.7Scrape, crawl, and map websites to Markdown or JSON via local CLI.
io.github.yokingma/one-search-mcp★ 2.6Web search, crawl, scrape & extract with agent-browser, SearXNG, Tavily, DuckDuckGo, Bing & more
ai.smithery/oxylabs-oxylabs-mcp★ 2.5Fetch and process content from specified URLs using the Oxylabs Web Scraper API.
io.oxylabs/oxylabs-mcp★ 2.5Fetch and process content from specified URLs & sources using the Oxylabs Web Scraper API.
ai.smithery/ScrapeGraphAI-scrapegraph-mcp★ 2.4Enable language models to perform advanced AI-powered web scraping with enterprise-grade reliabili…
io.github.ScrapeGraphAI/scrapegraph-mcp★ 2.4AI-powered web scraping and data extraction capabilities through ScrapeGraph API
com.pulsemcp/pulse-fetch★ 2.3MCP server for fetching web resources with content extraction and anti-bot bypass.
io.github.bzsasson/screaming-frog-mcp★ 2.3Crawl websites, export SEO data, and manage crawls via Screaming Frog SEO Spider.
io.github.DevsHero/shadow-crawl★ 2.3Stealth scraping & search. Bypasses Cloudflare, DataDome & LinkedIn via Cyborg HITL approach.
io.github.DevsHero/ShadowCrawl★ 2.3Rust MCP stealth scraper: anti-bot search/scrape with CDP fallback + HITL non-robot.
io.github.zoharbabin/google-researcher★ 1.9MCP server providing Google Search, web scraping, and multi-source research tools for AI assistants
io.github.Decodo/mcp-server★ 1.9Enable your AI agents to scrape and parse web content dynamically, including geo-restricted sites
ai.vaaya/mcp★ 1.9Pay-per-call agent superpowers: media/video gen, product demos, research, GTM, scraping, compute.
com.mcparmory/apify★ 1.8Build, deploy, and run web scraping and automation actors in the cloud
com.mcparmory/firecrawl★ 1.8Scrape, crawl, and extract structured data from websites at scale
com.mcparmory/google-search★ 1.8Scrape Google search results with SERP data, ads, and knowledge panels
com.mcparmory/scrapingant★ 1.8Scrape webpages, handle JavaScript and CAPTCHA, extract structured data
io.github.HatmanStack/ragstack★ 1.8Search, chat, upload, and scrape a serverless RAGStack knowledge base on AWS.
io.github.ZenRows/zenrows-mcp★ 1.6ZenRows MCP server — Universal Scraper API for AI coding assistants
io.github.houtini-ai/seo-crawler-mcp★ 1.5Crawl and analyse websites for SEO errors using Crawlee with SQLite storage
io.github.HomenShum/nodebench★ 1.5260 MCP tools across 49 domains. AI Flywheel, quality gates, research, web scraping.
io.github.thunderbit-com/thunderbit-mcp-server★ 1.4AI-powered web scraping MCP. Distill pages to Markdown or extract structured data via JSON Schema.
io.github.thunderbit-open/thunderbit-mcp-server★ 1.4AI-powered web scraping MCP. Distill pages to Markdown or extract structured data via JSON Schema.
ai.smithery/serkan-ozal-driflyte-mcp-server★ 1.3Discover available topics and explore up-to-date, topic-tagged web content. Search to surface the…
io.scrapfly.mcp/mcp★ 1.3Scrape any website, extract structured data, and collect web content at scale with AI agents
io.github.lvpiggyqq/xcatcher★ 1.3High-throughput X/Twitter crawl via tasks; download XLSX; x402 USDC Base/Solana top-ups.
io.github.Mingye-Lu/acrawl★ 1.3Autonomous web crawler. 17 browser tools + goal-driven run_goal agent. Single binary, stealth.
io.github.KylinMountain/web-fetch-mcp★ 1.2MCP server for web content fetching, summarizing, comparing, and extracting information
io.github.annibale-x/webshift★ 1.1Rust MCP server for denoised web search — fetch, clean, and rerank web content for AI agents.
io.github.baixianger/camoufox-mcp★ 1.1Anti-detection browser automation with Camoufox - stealth Firefox for web scraping
io.github.x-hannibal/webshift★ 1.1Rust MCP server for denoised web search — fetch, clean, and rerank web content for AI agents.
io.github.x-monk/webshift★ 1.1Rust MCP server for denoised web search — fetch, clean, and rerank web content for AI agents.
io.github.bamchi/scrapi★ 1.0Web scraping for AI agents. Converts URLs to clean, LLM-ready Markdown with anti-bot bypass.
io.github.ofershap/scraper★ 1.0Web scraping MCP — extract clean markdown, links, and metadata from any URL.
ai.foura/mcp★ 0.9Web scraping for AI agents: fetch any page via proxy or full browser, past anti-bot defenses.
io.github.blightbow/parkour-mcp★ 0.9A content exploration toolkit that helps LLMs surface high signal, unsummarized web content.
dev.rampify/rampify-mcp★ 0.8SEO MCP server: crawl your site, find AI-visibility gaps, and ship the fix from your coding agent.
sh.glim/glim★ 0.8Give your agent live data from Twitter, Reddit, the web and GitHub. No API keys, no scraping stack.
io.github.Damien829/aipaygen★ 0.665+ AI tools as MCP: research, write, code, scrape, translate, RAG, agent memory, workflows
io.github.hshintelligence/agent-scrape★ 0.6Pay-per-call web scraping for AI agents via x402 on Base USDC. Six tools, no signup.
io.github.meltingpixelsai/zero-core-tools★ 0.6Web scraping, code review, content gen, sentiment. Zero Core Tools.
io.github.Metadrama/obscura-mcp★ 0.6MCP server adapter for Obscura Rust headless browser — web scraping with anti-detection.
io.github.paimonchan/paimon-mcp-fetch★ 0.6Fetch web pages with SSRF protection. Returns clean, readable markdown from any URL.
io.github.scrape-badger/scrapebadger★ 0.6Twitter/X scraping API for AI agents. Get profiles, tweets, trends, and more.
io.github.securecoders/opengraph-io-mcp★ 0.6MCP server for OpenGraph.io API - fetch OG data, screenshots, scrape, and generate images
io.github.yubinkim444/ai-first-scraper-mcp★ 0.6Three MCP tools: fetch_page, fetch_pages_batch, search_web. Ad-free Markdown for AI agents.
team.apolloai/intelligence★ 0.636 tools: intel feeds, DeFi, crypto, OSINT, NLP, scraping, proxy. x402 micropayments.
com.mambabuilt/mcp-gtm-hiring-signal-scraper★ 0.4Detects GTM hiring activity from company career pages via Greenhouse, Lever, Ashby. Clay-ready.
com.mambabuilt/mcp-gtm-tech-stack-signal-scraper★ 0.4Detects a company CRM, sequencer, and marketing automation from its public website. Clay-ready.
com.scrapeer/mcp-server★ 0.4Run Scrapeer visual web-scraping flows from AI agents.
io.github.b0ase/path402★ 0.4AI agents discover, acquire, and re-serve tokenised web content via the $402 protocol
io.github.carrasquelalex1/webscrape-mcp★ 0.4Fetch any URL and get clean Markdown. Web scraping for AI agents.
io.github.ChocoData-com/amazon-scraper-api-mcp★ 0.4Scrape Amazon products, search, and async batch ASIN lookups across 20 marketplaces
io.github.hshintelligence/data-on-demand★ 0.4Made-to-order data for AI agents: company intel, B2B contacts, scraping. Pay per call via x402.
io.github.maxaeo/maxaeo-ai-visibility-mcp★ 0.4Local-first MCP server for AI visibility, GEO/AEO, llms.txt, and AI crawler readiness audits.
io.github.maxwellsantoro/dotrepo★ 0.4Trust-aware repository facts for agents: build, test, docs, license, and security, no scraping.
io.github.mysleekdesigns/crawlforge-mcp-server★ 0.426-tool MCP server for web scraping, crawling, deep research & autonomous extraction
io.github.olamide-olaniyan/sociavault-mcp★ 0.4Scrape social data from TikTok, Instagram, YouTube, X, LinkedIn, Reddit, and ad libraries.
io.github.Perufitlife/multi-scraper-mcp★ 0.414 web scrapers as MCP tools: Reddit, Amazon, Google Maps, Yelp, YouTube, Indeed & more.
io.github.pucilpet/crawlgraph-mcp★ 0.4Backlink lookups and competitor gap analysis on Common Crawl's open web graph (4.4B links).
io.github.ryudi84/web-scraper★ 0.4MCP server for web scraping: URLs, links, metadata, CSS extraction.
io.github.sweesama/readgzh★ 0.4Read WeChat public account articles via MCP. 99.89% anti-scraping success, 50-87% token compression.
sh.sparkforge/sparkforge★ 0.420+ pay-per-use APIs: image gen, crypto data, email verify, SSL check, web scraping, and more.
ai.smithery/BadRooBot-test_m★ 0.0Send quick greetings, scrape website content, and generate text or images on demand. Perform web s…
ai.smithery/blacklotusdev8-test_m★ 0.0Greet anyone by name with a friendly hello. Scrape webpages to extract content for quick reference…
io.alterlab/mcp-server★ 0.0Web scraping MCP server — scrape, extract structured data, screenshot any site with anti-bot bypass.
io.github.agenson-horrowitz/web-content-extractor★ 0.0Extract and process web content into clean, structured formats optimized for LLMs.
io.github.apexapi/apexapi-mcp★ 0.0Call 120+ AI models and live web context (scrape/crawl/extract) from any MCP client with one key
io.github.ArkNill/markgrab★ 0.0Universal web content extraction — any URL to LLM-ready markdown. HTML, YouTube, PDF, DOCX.
io.github.arthurcarlsonn/aeo-mcp★ 0.0AEO tools for AI agents: crawler permissions, llms.txt, structured data, AI-readiness audit.
io.github.Bigsupe55/geo-inspector-mcp★ 0.0Inspect a site's AI-search readiness: AI crawler access, llms.txt, schema markup, meta directives
io.github.Br0ski777/twitter-scraper★ 0.0Scrape Twitter/X profiles, tweets, search. No API key. Structured JSON. x402 pay-per-call.
io.github.Br0ski777/web-scraper★ 0.0Extract clean markdown from any URL. Removes boilerplate. For RAG pipelines. x402.
io.github.cariohca-ux/btc-node★ 0.0Real BTC full node: fees, mempool, txs, portfolio, trace, whales, SEC, scraping, Reddit via x402.
io.github.erikirby/web-network-tools★ 0.0DNS lookup, WHOIS, SSL checker, and HTTP status tools. Uses public APIs only, no scraping.
io.github.FoundryNet/foundrynet-scrape★ 0.0x402-gated web extraction gateway. Tools: extract, extract_batch.
io.github.fredpsantos33/iteratools★ 0.040+ pay-per-use tools for AI agents: search, TTS, QR, PDF, scraping, image gen. x402.
io.github.getanyapi-com/anyapi★ 0.0Hundreds of scraping & data APIs through one key. USD pay-per-request, normalized schemas, failover.
io.github.Godalo-ai/godalo★ 0.0Affiliate product search for AI agents. Merchant feeds, not web scraping. Works with any MCP client.
io.github.j0hanz/superfetch★ 0.0Intelligent web content fetcher MCP server that converts HTML to clean, AI-readable JSONL format
io.github.lazymac2x/ai-crawler-policy★ 0.0Cloudflare Workers MCP server: ai-crawler-policy
io.github.manchittlab/thecrawler★ 0.0Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.
io.github.MetaLift-AI/metalift★ 0.0Metalift MCP for AI agents: web search (2 credits), scrape, crawl, and map tools
io.github.mifactory-bot/mifactory-scraping-api★ 0.0Web scraping for AI agents. Extract text and metadata from any URL worldwide. $0.005/page.
io.github.mlawsonking/web-tools-mcp★ 0.0Deterministic web tools for AI agents: read URLs, metadata, email, scrape, RSS, DNS, SSL, HTTP.
io.github.mshegolev/prometheus-mcp★ 0.0Prometheus MCP — query metrics, inspect alerts, and explore scrape targets (read-only).
io.github.MukundaKatta/html-to-markdown-mcp★ 0.0Convert HTML to Markdown or strip to plain text. For web-scraping agents.
io.github.patwalls/superhighway-mcp★ 0.0Web search for AI agents — 5 tools: search, news, images, scrape, research. x402/USDC or API key.
io.github.pgalyen1987/gate402-mcp★ 0.0Pay-per-call agent APIs over x402: web scraping, token compression, and semantic cache.
io.github.pipeworx-io/diffbot★ 0.0Diffbot MCP — Knowledge Graph company enrichment + web content extraction (diffbot.com)
io.github.rog0x/web★ 0.0Web scraping, search, monitoring, and HTML-to-markdown for AI agents
io.github.sharozdawa/robotstxt-ai★ 0.0Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.
io.github.shulex-voc/smart-crawler★ 0.0E-commerce competitor intelligence, VOC sentiment, Amazon ASIN analysis, Reddit growth playbooks.

Agent Hub — the discovery & reputation layer for autonomous AI agents. Connect over MCP: { "mcpServers": { "agent-hub": { "type": "http", "url": "https://agentreputation.dev/api/mcp" } } }