AI web scraping agent
Editorial picks for "ai web scraper".
32 tools
ScrapeGraphAI
LLM-driven web scraping API that turns natural-language prompts into structured JSON.
n8n
Source-available workflow automation with first-class AI-agent and RAG building blocks.
Skyvern
AI browser agent that automates web workflows from natural-language instructions, with CAPTCHA and 2FA handling built in.
Browser Use
Open-source browser automation harness and cloud platform for LLM agents that drive real websites.
Sim
Open-source visual workspace for building, deploying, and monitoring AI agents.
Exa
Web search API built for AI agents, with structured outputs and token-efficient highlights.
Firecrawl
Web scraping and crawling API that returns LLM-ready markdown, JSON, or structured data from any URL.
Manus
Generalist agent for research, code, and web tasks.
OpenCLI
Open-source bridge that hands your already-logged-in Chrome session to a CLI and AI agents.
Open Deep Research
Minimal open-source deep-research agent that iteratively searches, scrapes, and reasons to produce cited markdown reports.
Hyperbrowser
Cloud browser infrastructure built for AI agents that need to scrape, click, and navigate the live web.
PageAgent
An in-page JavaScript GUI agent that drives web interfaces with natural language, no headless browser required.
AgenticSeek
Open-source local-first AI agent that browses the web, writes code, and runs tasks without sending anything to the cloud.
Browser Use Web UI
Gradio web UI for running browser-use AI agents in a real or persistent Chrome session.
You.com
Web search and research APIs purpose-built for LLMs and AI agents.
Nekton AI
Plain-English workflow automation that generates and runs cloud code from your prompt.
Browserbase
Headless browser infrastructure for AI agents
Dia Browser
An AI-native browser from The Browser Company that turns tabs into an agentic assistant for knowledge workers.
DuckDuckGo MCP Server
MCP server that gives Claude and other LLMs DuckDuckGo web search plus URL content fetching, with no API key required.
Exa MCP Server
Web search, code search, and company research for AI assistants over MCP
Fellou
The world's first agentic browser that plans and executes multi-step web workflows for you.
Firecrawl MCP Server
MCP server that gives Claude, Cursor, and other AI agents native web scrape, crawl, search, and extract tools via Firecrawl.
Kagi MCP Server
Official Kagi Search MCP server: search and extract the web from any MCP-compatible agent.
MCP Fetch Server
Official Model Context Protocol reference server that lets any MCP-compatible LLM fetch web pages and read them as clean markdown.
Opera Neon
An agentic browser from Opera built to act on the web, not just render it.
Perplexity Comet
An agentic AI browser from Perplexity that reads, reasons over, and acts on the pages you're viewing.
Perplexity MCP Server
Official MCP server for Perplexity's Sonar search, research, and reasoning APIs
Playwright MCP
Microsoft's official Model Context Protocol server for browser automation via Playwright.
Puppeteer MCP Server
Reference MCP server that lets LLMs drive a real Chromium browser via Puppeteer.
Scrapling
Adaptive Python web-scraping framework with a built-in MCP server for AI coding agents.
Stagehand
The SDK for browser agents
Tavily
One secure API for real-time web access for AI agents