Web Scraper
Crawl and extract structured data from websites. Uses a tiered approach -- Zo built-ins for simple fetches, BeautifulSoup for custom parsing, Crawl4AI for deep crawls and LLM-powered extraction. Use when you need to scrape a website, extract structured data, or crawl multiple pages.
Install
npx skills add zocomputer/skills --skill web-scrapergit clone --depth 1 https://github.com/zocomputer/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/Community/web-scraper ~/.claude/skills/web-scraperPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
## CLI ```bash python3 /home/workspace/Skills/web-scraper/scripts/scraper.py <command> [options] ``` | Command | Description | |---------|-------------| | `scrape <url>` | Fetch page content as markdown | | `scrape <url> --css <selector>` | Extract elements matching a CSS selector | | `scrape <url> --schema <json-file>` | Extract structured data using a JSON CSS schema | | `scrape <url> --llm-extract "<instruction>"` | Use LLM to extract structured data (Crawl4AI) | | `crawl <url> --depth N --max-pages N` | Deep crawl following links (BFS) | | `batch <file> [--css <selector>]` | Scrape multiple URLs from a file (one per line) | ## Options | Flag | Description | |------|-------------| | `--output <file>` | Save output to file instead of stdout | | `--format json\|md\|text` | Output format (default: md for scrape, json for extract) | | `--js` | Force JavaScript rendering via Playwright | | `--wait <seconds>` | Wait after page load before extracting | | `--timeout <seconds>` | Request timeout (default: 30) | ## Tiered Approach The scraper automatically picks the lightest tool that can do the job: 1. **Simple fetch** (`scrape <url>`) -- Uses httpx + BeautifulSoup. Fast, no browser overhead. 2. **CSS extraction** (`--css` or `--schema`) -- BeautifulSoup with selectors. Still fast. 3. **JS rendering** (`--js` flag, or auto-fallback) -- Playwright via Crawl4AI. For SPAs and dynamic content. 4. **Deep crawl** (`crawl` command) -- Crawl4AI with BFS link following.
Details
- Skill name
web-scraper- Repository
- zocomputer/skills
- Path
Community/web-scraper/SKILL.md- License
- MIT
- Repo updated
- 29 July 2026
FAQ
What does the Web Scraper skill do?
Crawl and extract structured data from websites. Uses a tiered approach -- Zo built-ins for simple fetches, BeautifulSoup for custom parsing, Crawl4AI for deep crawls and LLM-powered extraction. Use when you need to scrape a website, extract structured data, or crawl multiple pages.
How do I install the Web Scraper skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add zocomputer/skills --skill web-scraper. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Web Scraper popular?
It lives in zocomputer/skills, which has 46 GitHub stars and 56 forks.
Related skills
Add Flashcards
46zocomputer/skills
Converts webpages, articles, or documents into study flashcards that you can review with spaced repetition
Blog Site
46zocomputer/skills
Build a blog or newsletter to share your thoughts.
Blogwatcher
46zocomputer/skills
Monitor blogs and RSS/Atom feeds for updates using the blogwatcher CLI.
09 Consistency Validation
44ChenXiangYpily1234/paper-storycraft-skills
Use for independent full-paper or pre-submission audits of claims, references, formulas, statistics, cross-references, figures and document integrity. Review only unless edits are explicitly requested.
Status
40ArogyaReddy/alirezarezvani-claude-skills
Memory health dashboard showing line counts, topic files, capacity, stale entries, and recommendations.
So What Generator
33AnugamChakra/think-like-a-strategy-consultant
Turns an observation, data point, chart, or finding into a real insight — the implication and the action — instead of just restating what happened. Use this skill whenever someone shares a fact, metric, chart, or table and needs to know what it means, asks "so what", "what's the takeaway", "why does this matter", "what should we do about this", or is presenting findings that read as observations rather than conclusions. Trigger it whenever a user has data but hasn't extracted the decision from it, even if they don't use the phrase "so what". This is the instinct that turns a background slide i
Browse the most popular Claude skills, all data & analytics skills, or write your own with the skills generator.