crawl4ai
pipStandard🕷️ A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents. Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools.
2630Updated 6 months ago
Overview
🕷️ A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents. Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools.
Installation
1Install the package
pip
pip install crawl4ai-mcp-server2Add to Claude
Add this configuration to your claude_desktop_config.json:
claude_desktop_config.json
{
"mcpServers": {
"sadiuysal-crawl4ai-mcp-server-github": {
"command": "uvx",
"args": [
"pip install crawl4ai-mcp-server"
]
}
}
}3Verify installation
Restart Claude Desktop, then ask:
"What tools do you have available from crawl4ai?"
Prerequisites
- Python 3.10 or higher
- pip or uv package manager
- Claude Desktop or Claude API access
- API key for crawl4ai
Configuration
API Key Required
This server requires an API key from crawl4ai. Add it to your environment or config.
| Variable | Required | Description |
|---|---|---|
| CRAWL4AI_API_KEY | Yes | Your crawl4ai API key |
Usage Examples
List Resources
"What resources are available in crawl4ai?"
Claude will query available resources and return a list of what you can access.
Get Details
"Show me details about [specific item] in crawl4ai"
Claude will fetch and display detailed information about the requested item.
Create New
"Create a new [item] in crawl4ai with [details]"
Claude will use the appropriate tool to create the resource and confirm success.
Works with crawl4ai
Related skills
Browse all skillsNeed help with MCP integration?
We build custom MCP integrations for B2B companies. From simple connections to complex multi-tool setups.