maw
🥈SilverCrawl any website into a single searchable file. Query it offline forever. Useful for operations teams needing persistent access to web data without relying on live connections. Integrates with Claude agents for automated data extraction and analysis.
3520Updated 2mo ago
Intermediate30min to implementautomation
Saves ~60 min per use
Quick InstallView Source
git clone https://github.com/memvid/maw.gitWorks with:
Claude
Overview
About This Skill
maw crawls websites, documentation, blogs, codebases, and repositories into a single .mv2 file for persistent offline access. It automatically detects the right crawling method—from simple fetches to stealth browser mode for protected sites—and supports both keyword search (BM25) and semantic search with embeddings. Users can query crawled content instantly with keyword lookup or ask AI-powered questions using Claude or OpenAI. Built for operations teams, developers, and researchers who need reliable access to web data without relying on live connections or maintaining browser tabs.
How to Use
Install globally with `npm i -g @memvid/maw` or use `npx @memvid/maw` without installing. Crawl a site with `maw https://example.com` to create a .mv2 file. Search with `maw find docs.mv2 "term"` for keyword search or `maw ask docs.mv2 "question?"` for AI answers (requires OPENAI_API_KEY). Control crawl depth and page limits with `--depth` and `--max-pages` flags.
Use Cases
Archive documentation for offline reference and persistent access
Search across multiple documentation sites combined into one file
Extract and analyze data from websites using AI agents
Crawl codebases and repositories for code analysis
Tags
Setup & Installation
Quick Install
No install command available. Check the GitHub repository for manual installation instructions.
Alternative Install (Git Clone)
git clone https://github.com/memvid/mawRequirements
- Claude Code or compatible AI agent
- Works with: Claude
Quick Start Guide
1
Install the Skill
Copy the install command above and run it in your terminal.
2
Open Your AI Agent
Launch Claude Code, Cursor, or your preferred AI coding agent.
3
Try It Out
Use the prompt template or examples below to test the skill.
4
Customize
Adapt the skill to your specific use case and workflow.
Usage Examples
Prompt Template
Crawl the website [WEBSITE_URL] and extract all relevant content into a single searchable file. Focus on sections related to [TOPIC]. Ensure the file is structured for easy querying and offline access. Include metadata such as timestamps, authors, and any other relevant details.
Example Output
The website [WEBSITE_URL] has been successfully crawled and the content has been extracted into a single searchable file. The file includes all relevant sections related to [TOPIC], with metadata such as timestamps, authors, and other relevant details. The file is structured for easy querying and offline access, making it useful for operations teams needing persistent access to web data without relying on live connections. The file can be integrated with Claude agents for automated data extraction and analysis.
Apply to these tools
Browse all toolsIronCalc
IronCalc is a spreadsheet engine and ecosystem
Microsoft Teams
Get more done every day with Microsoft Teams – powered by AI
ServiceNow
Enterprise workflow automation and service management platform
GPT for work
Automate your spreadsheet tasks with AI power
Respell
Agentic AI Workflow platform
Notion
Connected workspace for docs, wikis, and projects
Compatible MCP servers
Browse all MCP servers
Find the right skills for your stack
Take a free 3-minute scan and get personalized AI skill recommendations.
Take free scan