Scrapy Extra

Read this BEFORE writing, modifying, debugging, or explaining any Scrapy or web-poet code — spiders, page objects, pipelines, settings, selectors — down to a one-line fix, or scaffolding/configuring a Scrapy project, scrapy-poet, scrapy-zyte-api, or browser rendering. These libraries moved on since your training cutoff; invoke this FIRST. NOT for building a scraper for a given site (scrape), a saved HTML file (scrape-analyze-page), or Scrapy Cloud deploy/jobs (zyte).

33 stars 5 forksData & Analytics Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add zytedata/claude-skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add zytedata/claude-skills --skill scrapy-extra
Manual install
git clone --depth 1 https://github.com/zytedata/claude-skills.git /tmp/claude-skills
mkdir -p ~/.claude/skills
cp -r /tmp/claude-skills/skills/scrapy-extra ~/.claude/skills/scrapy-extra

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

This file is 171 lines long; read all of them.

You are doing hands-on Scrapy coding: writing or changing spiders, page
objects, items, pipelines, settings, or an entire project.

For a **new** spider, prefer web-poet page objects — isolating extraction
keeps the spider focused on crawling and makes it easier to maintain, test,
and reuse. How far to go depends on the project:

- Brand-new project, or one already configured with scrapy-poet: use page
  objects unless told otherwise.
- Existing project that has spiders but no scrapy-poet: suggest adding
  scrapy-poet support (unless the user has said they don't want it) rather
  than assuming it; a plain spider that yields dicts or items is the fallback.

Never migrate an existing spider to page objects, or add web-poet to a
project that doesn't use it, unless asked.

Don't reimplement what Scrapy already does: a spider needs no seen-URL set,
since the dupefilter drops duplicate requests on its own.

There is no fixed workflow, no required inputs or outputs. Read the task, read
the relevant reference(s) below, and write idiomatic code.

## References

Bundled in `${CLAUDE_SKILL_DIR}/references/` — read only the ones the task
calls for, matched by what you are about to write; most tasks need one:

- Writing or debugging a spider, item, pipeline, selector, feed export or
  setting → `scrapy.md`.
- Writing or debugging a page object, or saving and running its fixtures →
  `web-poet.md`.
Read the full skill on GitHub

Details

Skill name
scrapy-extra
Path
skills/scrapy-extra/SKILL.md
Repo updated
26 October 2026

FAQ

What does the Scrapy Extra skill do?

Read this BEFORE writing, modifying, debugging, or explaining any Scrapy or web-poet code — spiders, page objects, pipelines, settings, selectors — down to a one-line fix, or scaffolding/configuring a Scrapy project, scrapy-poet, scrapy-zyte-api, or browser rendering. These libraries moved on since your training cutoff; invoke this FIRST. NOT for building a scraper for a given site (scrape), a saved HTML file (scrape-analyze-page), or Scrapy Cloud deploy/jobs (zyte).

How do I install the Scrapy Extra skill?

Claude Code plugin: /plugin marketplace add zytedata/claude-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Scrapy Extra popular?

It lives in zytedata/claude-skills, which has 33 GitHub stars and 5 forks.

Related skills

Scrape

33

zytedata/claude-skills

Build a full-site or multi-page Scrapy spider to get structured data from a website, in a new project or added to an existing one. Do not use for analyzing locally saved HTML, fixing or debugging an existing spider, schema-only or existing-spec work, Scrapy Cloud operations, or advisory questions.

Data & Analytics

Scrape Analyze Page

33

zytedata/claude-skills

Extract structured data (all available fields with values) from a page saved locally as an HTML file, optionally following a schema. Do not invoke when the user provides a URL. When invoking, pass the user's full request verbatim as args — do not pre-parse file paths and don't rephrase it.

Data & Analytics

Scrape Plan

33

zytedata/claude-skills

Plan a web scrape and author a validated extraction spec — explore a detail page, discover fields, confirm the schema, then validate it with more pages, HTML-variant comparison, extracted values, and an optional browser review, and present the plan for approval.

Data & Analytics

Add Analytics Instrumentation

48

amplitude/mcp-marketplace

End-to-end analytics instrumentation workflow for a PR, branch, file, directory, or feature. Reads the code, discovers what events should be tracked, and produces a concrete instrumentation plan — all in one shot. Use this skill whenever a user wants to add analytics to a PR, asks "instrument this PR", "add tracking to this branch", "what analytics does this file need", "instrument the checkout flow", "run the full instrumentation workflow", or any request that implies going from code changes to a tracking plan. Also trigger when the user gives you a PR link, branch name, file path, or feature

Data & Analytics12

Analyze Chart

48

amplitude/mcp-marketplace

Performs deep analysis of a specific Amplitude chart to explain trends, anomalies, and likely drivers. Use when a metric looks unusual, investigating a spike or drop, or understanding the "why" behind numbers.

Data & Analytics12

Analyze Dashboard

48

amplitude/mcp-marketplace

Deeply analyze Amplitude dashboards by analyzing key charts, surfacing top areas for concern and takeaways, identify anomalies, then explain changes using customer feedback trends

Data & Analytics12

Browse the most popular Claude skills, all data & analytics skills, or write your own with the skills generator.