The mcp-smart-crawler server enables AI agents to crawl web content using Playwright. It exposes commands to navigate pages, extract data, and handle dynamic content. It connects to websites and web applications. Developers use it to gather training data for AI models, automate web scraping tasks, and monitor web content changes.
The mcp-smart-crawler server enables AI agents to crawl web content using Playwright. It exposes commands to navigate pages, extract data, and handle dynamic content. It connects to websites and web applications. Developers use it to gather training data for AI models, automate web scraping tasks, and monitor web content changes.
npx -y @mseep/mcp-smart-crawlerAdd this configuration to your claude_desktop_config.json:
{
"mcpServers": {
"mseep-mcp-smart-crawler-npm": {
"command": "npx",
"args": [
"-y",
"npx -y @mseep/mcp-smart-crawler"
]
}
}
}Restart Claude Desktop, then ask:
"What tools do you have available from mcp-smart-crawler?"
API Key Required
This server requires an API key from mcp-smart-crawler. Add it to your environment or config.
| Variable | Required | Description |
|---|---|---|
| MCP-SMART-CRAWLER_API_KEY | Yes | Your mcp-smart-crawler API key |
"What resources are available in mcp-smart-crawler?"
Claude will query available resources and return a list of what you can access.
"Show me details about [specific item] in mcp-smart-crawler"
Claude will fetch and display detailed information about the requested item.
"Create a new [item] in mcp-smart-crawler with [details]"
Claude will use the appropriate tool to create the resource and confirm success.
See what tools in your stack can connect to AI.
We build custom MCP integrations for B2B companies. From simple connections to complex multi-tool setups.