Rotate Scraping Proxies
Escalate blocked scraping campaigns with provider-neutral proxy rotation — pick datacenter/residential/mobile pools, integrate rotation with scrapling, configure session stickiness for stateful flows, monitor cost and health, stay inside legal and ethical limits. Use after `headless-web-scraping` client-side stealth (StealthyFetcher, rate limiting, robots.txt) is insufficient and traffic is legitimate.
Install
npx skills add pjt222/agent-almanac --skill rotate-scraping-proxiesgit clone --depth 1 https://github.com/pjt222/agent-almanac.git /tmp/agent-almanac
mkdir -p ~/.claude/skills
cp -r /tmp/agent-almanac/i18n/caveman-lite/skills/rotate-scraping-proxies ~/.claude/skills/rotate-scraping-proxiesPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Rotate Scraping Proxies Network-layer escalation for scraping where client-side stealth is exhausted. Proxy rotation is a last resort, not a default — expensive, ethically charged, and easily misused. This skill teaches when *not* to use it as much as how. ## When to Use - `headless-web-scraping` (Fetcher → StealthyFetcher → DynamicFetcher) tried and target still returns 403/429/geo-blocks - Rate limiting already at 3+ second intervals and `robots.txt` permits the path - User-Agent and TLS fingerprint already realistic (not default `python-requests`) - Scraping is legitimate: public data, no auth circumvention, no paywall bypass, no personal data harvested without legal basis - You can budget proxy traffic and accept operational complexity **Do not use** when: a public API exists (use it), the site's ToS forbids automated access, you would circumvent geo-licensing, or the goal is fraud / credential stuffing / sneaker bots / content piracy. ## Inputs - **Required**: Target URLs and the legal basis for scraping them - **Required**: Proxy pool credentials (read from environment, never hard-coded) - **Required**: Pool type — datacenter, residential, or mobile - **Optional**: Geographic targeting (country / region / city) - **Optional**: Rotation granularity — per-request (default) or sticky session - **Optional**: Daily traffic / spend cap - **Optional**: Rate limit delay in seconds (default: 1, even with rotation) ## Procedure
Details
- Skill name
rotate-scraping-proxies- Repository
- pjt222/agent-almanac
- Path
i18n/caveman-lite/skills/rotate-scraping-proxies/SKILL.md- License
- MIT
- Repo updated
- 18 November 2026
- Listed in
- BehiSecc/awesome-claude-skills
FAQ
What does the Rotate Scraping Proxies skill do?
Escalate blocked scraping campaigns with provider-neutral proxy rotation — pick datacenter/residential/mobile pools, integrate rotation with scrapling, configure session stickiness for stateful flows, monitor cost and health, stay inside legal and ethical limits. Use after `headless-web-scraping` client-side stealth (StealthyFetcher, rate limiting, robots.txt) is insufficient and traffic is legitimate.
How do I install the Rotate Scraping Proxies skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add pjt222/agent-almanac --skill rotate-scraping-proxies. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Rotate Scraping Proxies popular?
It lives in pjt222/agent-almanac, which has 45 GitHub stars and 4 forks, and is included in 1 curated skill list.
Related skills
Adapt Architecture
45pjt222/agent-almanac
Execute structural metamorphosis using strangler fig migration, chrysalis phases, and interface preservation. Covers transformation planning, parallel running, progressive cutover, rollback design, and post-metamorphosis stabilization for system architecture evolution. Use when assess-form has classified the system as READY for transformation, when migrating from monolith to microservices, when replacing a core subsystem while dependents continue operating, or when any architectural change must be gradual rather than big-bang.
Adaptic
45pjt222/agent-almanac
Master skill composing the 5-step synoptic cycle for panoramic synthesis across multiple domains. Orchestrates meditate, expand-awareness, observe, awareness, integrate-gestalt, and express-insight into a coherent process that produces unified understanding rather than sequential compromise. Use when a problem genuinely spans 3+ domains and the interactions between domains matter more than depth in any one, when sequential analysis feels like compromise rather than integration, or before major architectural decisions affecting multiple stakeholders.
Add Puzzle Type
45pjt222/agent-almanac
Scaffold a new puzzle type across all 10+ pipeline integration points in jigsawR. Creates the core puzzle module, wires it into the unified pipeline (generation, positioning, rendering, adjacency), adds ggpuzzle geom/stat layers, updates DESCRIPTION and config.yml, extends the Shiny app, and creates a comprehensive test suite. Use when adding a completely new puzzle type to the package or following the 10-point integration checklist to ensure nothing is missed end-to-end.
Scaffold Shiny App
45pjt222/agent-almanac
Scaffold a new Shiny application using golem (production R package), rhino (enterprise), or vanilla (quick prototype) structure. Covers framework selection, project initialization, and first module creation. Use to start a new interactive web app in R, create a dashboard or data explorer prototype, set up a production Shiny app as an R package with golem, or bootstrap an enterprise Shiny project with rhino.
Review Data Analysis
45pjt222/agent-almanac
Review a data analysis for quality, correctness, and reproducibility. Covers data quality assessment, assumption checking, model validation, data leakage detection, and reproducibility verification. Use when reviewing a colleague's analysis before publication, validating an ML pipeline before production deployment, auditing a report for regulatory or business decision-making, or performing a second-analyst review in a regulated environment.
Simulate Stochastic Process
45pjt222/agent-almanac
Simulate stochastic processes (Markov chains, random walks, SDEs, MCMC) with convergence diagnostics, variance reduction, and visualization. Use when generating sample paths for estimation, prediction, or visualization; when analytical solutions are intractable; running Monte Carlo estimation needing convergence guarantees; validating analytical results against empirical simulation; or sampling from complex posteriors via MCMC.
Browse the most popular Claude skills, all data & analytics skills, or write your own with the skills generator.