Web Scraper
Extract structured data from websites. Use when: collecting competitor pricing; scraping product listings; extracting contact information; gathering research data; monitoring website changes
Install
npx skills add guia-matthieu/clawfu-skills --skill web-scrapergit clone --depth 1 https://github.com/guia-matthieu/clawfu-skills.git /tmp/clawfu-skills
mkdir -p ~/.claude/skills
cp -r /tmp/clawfu-skills/skills/automation/web-scraper ~/.claude/skills/web-scraperPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Web Scraper > Extract structured data from websites using BeautifulSoup and requests - turn any webpage into usable data. ## When to Use This Skill - **Competitor research** - Scrape pricing, features, positioning - **Lead generation** - Extract contact info from directories - **Content audit** - Pull headings, links, meta data - **Price monitoring** - Track competitor pricing changes - **Data collection** - Gather research data from multiple sources ## What Claude Does vs What You Decide | Claude Does | You Decide | |-------------|------------| | Structures analysis frameworks | Strategic priorities | | Synthesizes market data | Competitive positioning | | Identifies opportunities | Resource allocation | | Creates strategic options | Final strategy selection | | Suggests implementation approaches | Execution decisions | ## Dependencies ```bash pip install beautifulsoup4 requests pandas click lxml ``` ## Commands ### Scrape Elements ```bash python scripts/main.py scrape https://example.com --selector "h1,h2,p" python scripts/main.py scrape https://example.com --selector ".product-price" ``` ### Extract Links ```bash python scripts/main.py links https://example.com python scripts/main.py links https://example.com --internal-only ``` ### Extract Emails ```bash python scripts/main.py emails https://example.com python scripts/main.py emails https://example.com --depth 2 ``` ### Extract Structured Data
Details
- Skill name
web-scraper- Repository
- guia-matthieu/clawfu-skills
- Path
skills/automation/web-scraper/SKILL.md- License
- MIT
- Repo updated
- 1 October 2026
- Listed in
- BehiSecc/awesome-claude-skills
FAQ
What does the Web Scraper skill do?
Extract structured data from websites. Use when: collecting competitor pricing; scraping product listings; extracting contact information; gathering research data; monitoring website changes
How do I install the Web Scraper skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add guia-matthieu/clawfu-skills --skill web-scraper. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Web Scraper popular?
It lives in guia-matthieu/clawfu-skills, which has 150 GitHub stars and 32 forks, and is included in 1 curated skill list.
Related skills
Aarrr Metrics
150guia-matthieu/clawfu-skills
Measure and optimize growth using the AARRR (Pirate Metrics) framework with stage-specific KPIs and funnel analysis
Ab Test Stats
150guia-matthieu/clawfu-skills
Calculate A/B test statistical significance. Use when: determining if test results are significant; calculating required sample size; estimating test duration; analyzing conversion experiments; making data-driven decisions
Account Health
150guia-matthieu/clawfu-skills
Assess customer account health using product usage, support sentiment, payment status, and relationship signals
Data Visualization
223NeverSight/learn-skills.dev
Data visualization with chart selection, color theory, and annotation best practices. Covers chart types (bar, line, scatter, heatmap), axes rules, and storytelling with data. Use for: charts, graphs, dashboards, reports, presentations, infographics, data stories. Triggers: data visualization, chart, graph, data chart, bar chart, line chart, scatter plot, data viz, visualization, dashboard chart, infographic data, data presentation, chart design, plot, heatmap, pie chart alternative
Twitter X Scraping
201swyxio/skills
Use when scraping public Twitter/X timelines or lists through Nitter-style mirrors with axios and Playwright fallback, including anti-bot challenge handling and pagination limits.
Fusion Data Analysis
223NeverSight/learn-skills.dev
Fusion plasma data analysis across the tokamak/stellarator research pipeline. Use when the user asks to analyze, interpret, or QA equilibrium reconstructions (EFIT, LIUQE, G-EQDSK, gfile, eqdsk, flux-surface geometry, safety factor q-profile, magnetic shear s, normalized flux ψ_N, ρ_tor, r/a, elongation κ, triangularity δ), kinetic profiles (electron/ion temperature T_e/T_i, electron density n_e, toroidal/impurity rotation v_φ/v_t, profile fitting with polynomial/tanh/mtanh/Gaussian, normalized gradient scale lengths R/L_T and R/L_n, profile alignment and interpolation, error propagation), tra
Browse the most popular Claude skills, all data & analytics skills, or write your own with the skills generator.