Web Scraping

This skill activates for web scraping and Actor development. It proactively discovers APIs via traffic interception, recommends optimal strategy (traffic interception/sitemap/API/DOM scraping/hybrid), and implements iteratively. For production, it guides TypeScript Actor creation via Apify CLI.

108 stars 22 forksData & Analytics

Install

skills CLI (Claude Code, Cursor, Codex and more)

npx skills add yfe404/web-scraper --skill web-scraping

Manual install

git clone --depth 1 https://github.com/yfe404/web-scraper.git /tmp/web-scraper
mkdir -p ~/.claude/skills
cp -r /tmp/web-scraper/. ~/.claude/skills/web-scraping

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Web Scraping with Intelligent Strategy Selection

## When This Skill Activates

Activate automatically when user requests:
- "Scrape [website]"
- "Extract data from [site]"
- "Get product information from [URL]"
- "Find all links/pages on [site]"
- "I'm getting blocked" or "Getting 403 errors" (loads `strategies/anti-blocking.md`)
- "Make this an Apify Actor" (loads `apify/` subdirectory)
- "Productionize this scraper"

## Input Parsing

Determine reconnaissance depth from user request:

| User Says | Mode | Phases Run |
|-----------|------|------------|
| "quick recon", "just check", "what framework" | Quick | Phase 0 only |
| "scrape X", "extract data from X" (default) | Standard | Phases 0-3 + 5, Phase 4 only if protection signals detected |
| "full recon", "deep scan", "production scraping" | Full | All phases (0-5) including protection testing |

Default is Standard mode. Escalate to Full if protection signals appear during any phase.

## Adaptive Reconnaissance Workflow

This skill uses an adaptive phased workflow with quality gates. Each gate asks **"Do I have enough?"** — continue only when the answer is no.

**See**: `strategies/framework-signatures.md` for framework detection tables referenced throughout.

### Phase 0: QUICK ASSESSMENT (curl, no browser)

Gather maximum intelligence with minimum cost — a single HTTP request.

**Step 0a: Fetch raw HTML and headers**
```bash
curl -s -D- -L "https://target.com/page" -o response.html
```

Read the full skill on GitHub

Details

Skill name: web-scraping
Repository: yfe404/web-scraper
License: MIT
Repo updated: 18 March 2026

FAQ

What does the Web Scraping skill do?

How do I install the Web Scraping skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add yfe404/web-scraper --skill web-scraping. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Web Scraping popular?

It lives in yfe404/web-scraper, which has 108 GitHub stars and 22 forks.

Related skills

Teachable Figures

111

ywEdAi/teachable-figures

Turn lessons, articles, chapters, or technical ideas into interactive teaching figures using predict → try → explain. Create reusable 2D widgets, working isometric SVG objects, mechanical assemblies and cutaways, exploded views, or true 3D turning models when spatial motion explains the concept. Use for explorable learning graphics, mechanism simulations, and mechanical or spatial explainers; retain the host app's lesson contracts and accessibility. Starts with a short guided interview unless the user asks to generate directly. Not for plain data charts, dashboards, slide decks, static illustr

Data & Analytics13

Fit Ride Studio

149

op7418/guizang-sports-skill

Analyze one or more cycling, running, or hiking FIT/KML activity tracks, generate evidence-based single or aggregate reports, open the bundled local Ride Relief report page when the environment supports it, enrich results with user-confirmed public location context, and create shareable PNG or H.264 MP4 route stories on request. Use whenever a user wants to analyze, compare, visualize, or export .fit or .kml activities, including requests that mention a workout but do not yet include track files.

Data & Analytics12

Venice Billing

160

veniceai/skills

Venice billing and usage analytics - GET /billing/balance, GET /billing/usage-history (keyset-paginated per-request ledger, JSON or CSV), and GET /billing/usage-analytics (aggregated by date/model/key). All require an ADMIN key. GET /billing/usage was sunset on 2026-09-16 and now always returns 410. Covers the DIEM/EARNED_CREDITS/BUNDLED_CREDITS/USD consumption priority and building dashboards. (Beta)

Data & AnalyticsIn 1 list27

Analytics Tracking Automation

145

jtrackingai/analytics-tracking-automation

Use when you need GA4 + GTM tracking delivery from site discovery through publish, or when the right phase entry point is still unclear.

Data & Analytics41

Easyplot

110

Rimagination/easyplot

Use when the user asks for scientific data preparation, exploratory analysis, statistical inference, model evaluation, or research figures in R or Python, including analysis-only requests, multi-panel composition, and publication export. EasyPlot is the user's single entry point for research data analysis and visualization; exclude unrelated business dashboards and general application development.

Data & Analytics

Tufte Data Viz

230

caylent/tufte-data-viz

Use when creating, reviewing, or styling charts, graphs, dashboards, sparklines, or any data visualization. Applies to Recharts, ECharts, Chart.js, matplotlib, Plotly, seaborn, D3.js, and SVG. Enforces Tufte principles (data-ink ratio, direct labeling, range-frame axes) plus modern screen-first standards (accessibility, responsive, dark mode).

Data & Analytics

Browse the most popular Claude skills, all data & analytics skills, or write your own with the skills generator.