civitai-web-scraper

🥇Gold

The civitai-web-scraper is a powerful tool designed to extract AI-generated images and prompts from Civitai. Built in Python, it enables users to efficiently gather creative content for various applications, enhancing their projects with unique visuals and ideas.

245250Updated 3mo ago

Intermediate30 minutes to implementresearch

Saves ~180 min per use

Quick InstallView Source

claude install ScreamingHawk/civitai-web-scraper

Works with:

GitHub Copilot

Overview

About This Skill

How to Use

1. **Install the civitai-web-scraper tool** - Clone the repository: `git clone https://github.com/[USER]/civitai-web-scraper.git` - Install dependencies: `pip install -r requirements.txt` - Ensure Python 3.8+ is installed. 2. **Run the scraper with your custom parameters** - Execute: `python civitai_scraper.py --categories "Cyberpunk Art", "Fantasy Creatures" --save-dir ./civitai_images` - Replace the categories with your target topics (e.g., "Anime Characters", "Landscapes"). 3. **Review the output** - Check the `civitai_images` directory for organized images by category. - Open `civitai_metadata.csv` to view extracted metadata in a structured format. 4. **Filter and refine results (optional)** - Use `--min-likes 500` to filter images with at least 500 likes. - Use `--resolution "1024x1024"` to limit results to specific resolutions. 5. **Automate for ongoing research** - Schedule the scraper to run weekly using cron jobs or task schedulers. - Combine with a tool like `pandas` to analyze trends in the metadata CSV over time. **Tips for Better Results:** - Use specific category names to avoid irrelevant results (e.g., "Anime Characters" instead of "Art"). - Combine categories with `--and` or `--or` flags for broader or narrower searches. - For large datasets, consider running the scraper in batches to avoid rate limits.

Use Cases

Collecting AI-generated visuals for marketing campaigns

Gathering prompts for creative writing

Analyzing trends in AI-generated content

Best For

GrowthMarketing

Setup & Installation

Quick Install

Terminal

claude install ScreamingHawk/civitai-web-scraper

Alternative Install (Git Clone)

git clone https://github.com/ScreamingHawk/civitai-web-scraper

Requirements

Claude Code or compatible AI agent
Works with: GitHub Copilot

Quick Start Guide

Install the Skill

Copy the install command above and run it in your terminal.

Open Your AI Agent

Launch Claude Code, Cursor, or your preferred AI coding agent.

Try It Out

Use the prompt template or examples below to test the skill.

Customize

Adapt the skill to your specific use case and workflow.

Usage Examples

Prompt Template

Extract all AI-generated images and their associated prompts from Civitai for the following categories: [CATEGORY_1], [CATEGORY_2], and [CATEGORY_3]. Save the images to a local directory named 'civitai_images' and organize them by category. Include metadata such as the model used, resolution, and likes in a CSV file named 'civitai_metadata.csv'.

Example Output

### Extracted Content from Civitai

**Category: Cyberpunk Art**

1. **Image:** `civitai_images/cyberpunk_art/001_cyberpunk_cityscape.png`
   - **Prompt:** "A neon-lit cyberpunk city at night, towering skyscrapers with holographic billboards, rain-slicked streets reflecting the glow, cinematic lighting, ultra-detailed, 8K resolution"
   - **Model Used:** Stable Diffusion XL 1.0
   - **Resolution:** 1024x1024
   - **Likes:** 1,247
   - **Tags:** cyberpunk, futuristic, neon, cityscape, rain, holographic

2. **Image:** `civitai_images/cyberpunk_art/002_cyberpunk_hacker.png`
   - **Prompt:** "A lone hacker in a dark room, wearing a futuristic visor, typing on a holographic keyboard, surrounded by floating code and data streams, cyberpunk aesthetic, highly detailed"
   - **Model Used:** SDXL 0.9
   - **Resolution:** 1152x896
   - **Likes:** 892
   - **Tags:** hacker, cyberpunk, futuristic, technology, dark

3. **Image:** `civitai_images/cyberpunk_art/003_cyberpunk_character.png`
   - **Prompt:** "A cybernetically enhanced mercenary with glowing cybernetic eyes, wearing a trench coat and gas mask, standing in a dystopian alley, cinematic lighting, ultra-realistic"
   - **Model Used:** Realistic Vision v5.1
   - **Resolution:** 768x1024
   - **Likes:** 2,103
   - **Tags:** mercenary, cybernetic, dystopian, character design, futuristic

**Category: Fantasy Creatures**

1. **Image:** `civitai_images/fantasy_creatures/001_dragon.png`
   - **Prompt:** "A majestic fire-breathing dragon soaring over a medieval castle, detailed scales, glowing eyes, epic fantasy scene, high fantasy art style"
   - **Model Used:** Juggernaut XL
   - **Resolution:** 1024x1024
   - **Likes:** 1,567
   - **Tags:** dragon, fantasy, medieval, fire, epic

2. **Image:** `civitai_images/fantasy_creatures/002_elf_archer.png`
   - **Prompt:** "An elven archer in a mystical forest, wearing leafy armor, aiming a glowing bow, surrounded by ancient trees and magical mist"
   - **Model Used:** Anything V5
   - **Resolution:** 896x1152
   - **Likes:** 987
   - **Tags:** elf, archer, fantasy, forest, mystical

**Metadata Summary (civitai_metadata.csv):**
```csv
Filename,Category,Prompt,Model,Resolution,Likes,Tags
001_cyberpunk_cityscape.png,Cyberpunk Art,"A neon-lit cyberpunk city at night...",Stable Diffusion XL 1.0,1024x1024,1247,"cyberpunk,futuristic,neon,cityscape,rain,holographic"
002_cyberpunk_hacker.png,Cyberpunk Art,"A lone hacker in a dark room...",SDXL 0.9,1152x896,892,"hacker,cyberpunk,futuristic,technology,dark"
003_cyberpunk_character.png,Cyberpunk Art,"A cybernetically enhanced mercenary...",Realistic Vision v5.1,768x1024,2103,"mercenary,cybernetic,dystopian,character design,futuristic"
001_dragon.png,Fantasy Creatures,"A majestic fire-breathing dragon...",Juggernaut XL,1024x1024,1567,"dragon,fantasy,medieval,fire,epic"
002_elf_archer.png,Fantasy Creatures,"An elven archer in a mystical forest...",Anything V5,896x1152,987,"elf,archer,fantasy,forest,mystical"
```

**Total Images Extracted:** 5
**Total Likes:** 6,796
**Most Popular Model:** Realistic Vision v5.1 (1 image, 2,103 likes)

Apply to these tools

Browse all tools

Unique

Enterprise AI for Finance

Compatible MCP servers

Browse all MCP servers

multiverse

76data-platforms

Find the right skills for your stack

Take a free 3-minute scan and get personalized AI skill recommendations.

Take free scan