The mcp-smart-crawler server enables AI agents to automate web content crawling using Playwright. It exposes commands to navigate, extract, and interact with web content. The server connects to websites and web applications, providing structured data for AI model training. Developers use it to gather web data for training datasets, web scraping tasks, and content analysis.
The mcp-smart-crawler server enables AI agents to automate web content crawling using Playwright. It exposes commands to navigate, extract, and interact with web content. The server connects to websites and web applications, providing structured data for AI model training. Developers use it to gather web data for training datasets, web scraping tasks, and content analysis.
npx -y mcp-smart-crawlerAdd this configuration to your claude_desktop_config.json:
{
"mcpServers": {
"mcp-smart-crawler-npm": {
"command": "npx",
"args": [
"-y",
"npx -y mcp-smart-crawler"
]
}
}
}Restart Claude Desktop, then ask:
"What tools do you have available from mcp-smart-crawler?"
API Key Required
This server requires an API key from mcp-smart-crawler. Add it to your environment or config.
| Variable | Required | Description |
|---|---|---|
| MCP-SMART-CRAWLER_API_KEY | Yes | Your mcp-smart-crawler API key |
"What resources are available in mcp-smart-crawler?"
Claude will query available resources and return a list of what you can access.
"Show me details about [specific item] in mcp-smart-crawler"
Claude will fetch and display detailed information about the requested item.
"Create a new [item] in mcp-smart-crawler with [details]"
Claude will use the appropriate tool to create the resource and confirm success.
See what tools in your stack can connect to AI.
We build custom MCP integrations for B2B companies. From simple connections to complex multi-tool setups.