AI web scraping agent
Editorial picks for "ai web scraper".
48 tools
ScrapeGraphAI
LLM-driven web scraping API that turns natural-language prompts into structured JSON.
Firecrawl
Web scraping and crawling API that returns LLM-ready markdown, JSON, or structured data from any URL.
CowAgent
Open-source, self-hosted AI agent framework that plans, uses tools, and executes multi-step tasks.
Stagehand
The SDK for browser agents
X402vps
Pay-per-hour Docker containers for autonomous AI agents, billed in USDC via the x402 protocol
Browser Use
Open-source browser automation harness and cloud platform for LLM agents that drive real websites.
Exa
Web search API built for AI agents, with structured outputs and token-efficient highlights.
OpenCLI
Open-source bridge that hands your already-logged-in Chrome session to a CLI and AI agents.
Open Deep Research
Minimal open-source deep-research agent that iteratively searches, scrapes, and reasons to produce cited markdown reports.
Hyperbrowser
Cloud browser infrastructure built for AI agents that need to scrape, click, and navigate the live web.
VisualWebArena
Open benchmark for evaluating multimodal web agents on realistic visual browsing tasks.
AgentGPT
Browser-based autonomous AI agent builder that chains GPT calls toward a goal you define.
Browser Use Web UI
Gradio web UI for running browser-use AI agents in a real or persistent Chrome session.
MindPal
No-code platform for building and orchestrating teams of AI agents that automate business workflows.
WebChatGPT
Browser extension that grafts live web search results onto ChatGPT (and Claude, Gemini) prompts.
Browserbase
Headless browser infrastructure for AI agents
DuckDuckGo MCP Server
MCP server that gives Claude and other LLMs DuckDuckGo web search plus URL content fetching, with no API key required.
Fellou
The world's first agentic browser that plans and executes multi-step web workflows for you.
Firecrawl MCP Server
MCP server that gives Claude, Cursor, and other AI agents native web scrape, crawl, search, and extract tools via Firecrawl.
Goose
Open-source, local-first AI agent for code, workflows, and everything in between.
Kagi MCP Server
Official Kagi Search MCP server: search and extract the web from any MCP-compatible agent.
Playwright MCP
Microsoft's official Model Context Protocol server for browser automation via Playwright.
Puppeteer MCP Server
Reference MCP server that lets LLMs drive a real Chromium browser via Puppeteer.
Scrapling
Adaptive Python web-scraping framework with a built-in MCP server for AI coding agents.
Suna
Open-source AI command center for deploying a workforce of autonomous agents across your company
Tavily
One secure API for real-time web access for AI agents
Claude
FeaturedAnthropic's flagship assistant for long-form writing, analysis, and coding.
Ernie Bot
Baidu's Mandarin-first ChatGPT rival, powered by the ERNIE model family
Moveworks
The enterprise AI assistant that searches, answers, and takes action across your business systems
Doubao
ByteDance's flagship AI assistant with a real agent mode and one of the largest user bases outside ChatGPT
Amp (Sourcegraph)
Frontier coding agent from Sourcegraph with pass-through model pricing
ChatGLM (Zhipu Qingyan)
Zhipu AI's bilingual ChatGLM assistant with agents, code, image and video generation
LangChain
The broad LLM application framework — chains, agents, retrievers.
n8n
Source-available workflow automation with first-class AI-agent and RAG building blocks.
Skyvern
AI browser agent that automates web workflows from natural-language instructions, with CAPTCHA and 2FA handling built in.
Claude Code
Anthropic's agentic coding tool that reads, edits, and ships code from the terminal, IDE, or web.
Hermes Agent
Open-source multi-channel AI agent with persistent memory, scheduling, and subagent delegation from Nous Research.
Higgsfield
AI video and image generation suite that aggregates 30+ frontier models under one workflow.
HyperFrames
Open-source video composition framework that lets AI agents build videos by writing HTML, CSS, and JS.
LibreChat
Open-source, self-hostable ChatGPT-style frontend that brings every major LLM provider under one roof.
Lovable
Chat-driven app builder that turns natural-language prompts into deployable full-stack web apps.
Poe
One subscription, every frontier model - Quora's multi-model chat hub with bots, image, video, and audio generation.
Berkeley Function-Calling Leaderboard
Open benchmark from UC Berkeley that ranks LLMs on real-world tool-use and function-calling accuracy.
Cube
Semantic layer that grounds LLM agents in your real business metrics instead of letting them hallucinate SQL.
ElevenLabs Conversational AI
Production-grade voice agent platform layering ElevenLabs TTS, ASR, and LLM orchestration into a single deployable stack.
Kimi
Moonshot AI's chat assistant with long-context document analysis, coding agents, and deep research built in.
Le Chat
Mistral's ChatGPT-style assistant, now an agent that browses, codes, and plugs into your work tools.
PyGPT
Open-source desktop AI assistant that wires every major LLM provider into one local app with agents, vision, and a code interpreter.