Intelligent web scraping Claude Code skill with automatic strategy selection and TypeScript-first Apify Actor development
git clone https://github.com/yfe404/web-scraper.gitIntelligent web scraping Claude Code skill with automatic strategy selection and TypeScript-first Apify Actor development
No install command available. Check the GitHub repository for manual installation instructions.
git clone https://github.com/yfe404/web-scraperCopy the install command above and run it in your terminal.
Launch Claude Code, Cursor, or your preferred AI coding agent.
Use the prompt template or examples below to test the skill.
Adapt the skill to your specific use case and workflow.
I need to scrape data from [WEBSITE_URL] for [COMPANY] in the [INDUSTRY] sector. The data I need includes [DATA_POINTS]. Please generate a TypeScript Apify Actor script that can handle [SPECIFIC_CHALLENGES] like pagination or dynamic content loading. Also, suggest the best scraping strategy for this website.
# Web Scraping Strategy for [WEBSITE_URL]
## Recommended Approach
- **Strategy**: Incremental crawling with request queue
- **Reasoning**: The website has dynamic content loading and requires handling pagination.
## TypeScript Apify Actor Script
```typescript
import { Actor } from 'apify'
const input = await Apify.getInput()
const requestQueue = await Apify.openRequestQueue()
// Add initial URL to the queue
await requestQueue.addRequest({ url: input.startUrl })
// Process each page
while (true) {
const request = await requestQueue.fetchNextRequest()
if (request === null) {
break
}
// ...
}
```
## Data Points to Extract
- [DATA_POINT_1]
- [DATA_POINT_2]
- [DATA_POINT_3]
## Handling Challenges
- **Dynamic Content**: Use Cheerio or Playwright for rendering.
- **Pagination**: Implement a recursive function to handle next page buttons.Real-time collaborative writing platform
Your one-stop shop for church and ministry supplies.
Automate your browser workflows effortlessly
Get real-time web data for your AI
AI assistant built for thoughtful, nuanced conversation
Automate your spreadsheet tasks with AI power
Take a free 3-minute scan and get personalized AI skill recommendations.
Take free scan