Rotate Scraping Proxies

Escalate blocked scraping campaigns with provider-neutral proxy rotation — pick datacenter/residential/mobile pools, integrate rotation with scrapling, configure session stickiness for stateful flows, monitor cost and health, stay inside legal and ethical limits. Use after `headless-web-scraping` client-side stealth (StealthyFetcher, rate limiting, robots.txt) is insufficient and traffic is legitimate.

45 stars 4 forksData & Analytics

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add pjt222/agent-almanac --skill rotate-scraping-proxies
Manual install
git clone --depth 1 https://github.com/pjt222/agent-almanac.git /tmp/agent-almanac
mkdir -p ~/.claude/skills
cp -r /tmp/agent-almanac/i18n/caveman-lite/skills/rotate-scraping-proxies ~/.claude/skills/rotate-scraping-proxies

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Rotate Scraping Proxies

Network-layer escalation for scraping where client-side stealth is exhausted.
Proxy rotation is a last resort, not a default — expensive, ethically charged,
and easily misused. This skill teaches when *not* to use it as much as how.

## When to Use

- `headless-web-scraping` (Fetcher → StealthyFetcher → DynamicFetcher) tried
  and target still returns 403/429/geo-blocks
- Rate limiting already at 3+ second intervals and `robots.txt` permits the path
- User-Agent and TLS fingerprint already realistic (not default `python-requests`)
- Scraping is legitimate: public data, no auth circumvention, no paywall bypass,
  no personal data harvested without legal basis
- You can budget proxy traffic and accept operational complexity

**Do not use** when: a public API exists (use it), the site's ToS forbids
automated access, you would circumvent geo-licensing, or the goal is fraud /
credential stuffing / sneaker bots / content piracy.

## Inputs

- **Required**: Target URLs and the legal basis for scraping them
- **Required**: Proxy pool credentials (read from environment, never hard-coded)
- **Required**: Pool type — datacenter, residential, or mobile
- **Optional**: Geographic targeting (country / region / city)
- **Optional**: Rotation granularity — per-request (default) or sticky session
- **Optional**: Daily traffic / spend cap
- **Optional**: Rate limit delay in seconds (default: 1, even with rotation)

## Procedure
Read the full skill on GitHub

Details

Skill name
rotate-scraping-proxies
Path
i18n/caveman-lite/skills/rotate-scraping-proxies/SKILL.md
License
MIT
Repo updated
18 November 2026

FAQ

What does the Rotate Scraping Proxies skill do?

Escalate blocked scraping campaigns with provider-neutral proxy rotation — pick datacenter/residential/mobile pools, integrate rotation with scrapling, configure session stickiness for stateful flows, monitor cost and health, stay inside legal and ethical limits. Use after `headless-web-scraping` client-side stealth (StealthyFetcher, rate limiting, robots.txt) is insufficient and traffic is legitimate.

How do I install the Rotate Scraping Proxies skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add pjt222/agent-almanac --skill rotate-scraping-proxies. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Rotate Scraping Proxies popular?

It lives in pjt222/agent-almanac, which has 45 GitHub stars and 4 forks, and is included in 1 curated skill list.

Related skills

Adapt Architecture

45

pjt222/agent-almanac

Execute structural metamorphosis using strangler fig migration, chrysalis phases, and interface preservation. Covers transformation planning, parallel running, progressive cutover, rollback design, and post-metamorphosis stabilization for system architecture evolution. Use when assess-form has classified the system as READY for transformation, when migrating from monolith to microservices, when replacing a core subsystem while dependents continue operating, or when any architectural change must be gradual rather than big-bang.

Software DevelopmentIn 1 list

Adaptic

45

pjt222/agent-almanac

Master skill composing the 5-step synoptic cycle for panoramic synthesis across multiple domains. Orchestrates meditate, expand-awareness, observe, awareness, integrate-gestalt, and express-insight into a coherent process that produces unified understanding rather than sequential compromise. Use when a problem genuinely spans 3+ domains and the interactions between domains matter more than depth in any one, when sequential analysis feels like compromise rather than integration, or before major architectural decisions affecting multiple stakeholders.

OtherIn 1 list

Add Puzzle Type

45

pjt222/agent-almanac

Scaffold a new puzzle type across all 10+ pipeline integration points in jigsawR. Creates the core puzzle module, wires it into the unified pipeline (generation, positioning, rendering, adjacency), adds ggpuzzle geom/stat layers, updates DESCRIPTION and config.yml, extends the Shiny app, and creates a comprehensive test suite. Use when adding a completely new puzzle type to the package or following the 10-point integration checklist to ensure nothing is missed end-to-end.

Testing & QAIn 1 list

Scaffold Shiny App

45

pjt222/agent-almanac

Scaffold a new Shiny application using golem (production R package), rhino (enterprise), or vanilla (quick prototype) structure. Covers framework selection, project initialization, and first module creation. Use to start a new interactive web app in R, create a dashboard or data explorer prototype, set up a production Shiny app as an R package with golem, or bootstrap an enterprise Shiny project with rhino.

Data & AnalyticsIn 1 list

Review Data Analysis

45

pjt222/agent-almanac

Review a data analysis for quality, correctness, and reproducibility. Covers data quality assessment, assumption checking, model validation, data leakage detection, and reproducibility verification. Use when reviewing a colleague's analysis before publication, validating an ML pipeline before production deployment, auditing a report for regulatory or business decision-making, or performing a second-analyst review in a regulated environment.

Data & AnalyticsIn 1 list

Simulate Stochastic Process

45

pjt222/agent-almanac

Simulate stochastic processes (Markov chains, random walks, SDEs, MCMC) with convergence diagnostics, variance reduction, and visualization. Use when generating sample paths for estimation, prediction, or visualization; when analytical solutions are intractable; running Monte Carlo estimation needing convergence guarantees; validating analytical results against empirical simulation; or sampling from complex posteriors via MCMC.

Data & AnalyticsIn 1 list

Browse the most popular Claude skills, all data & analytics skills, or write your own with the skills generator.