Twitter X Scraping
Use when scraping public Twitter/X timelines or lists through Nitter-style mirrors with axios and Playwright fallback, including anti-bot challenge handling and pagination limits.
Install
npx skills add swyxio/skills --skill twitter-x-scrapinggit clone --depth 1 https://github.com/swyxio/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/twitter-x-scraping ~/.claude/skills/twitter-x-scrapingPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Skill: Scraping Nitter.net for Tweets
This codebase scrapes [nitter.net](https://nitter.net) — an open-source Twitter frontend — to collect tweets from a list or profile within a time window. The technique works without a Twitter API key.
## Core Methodology
### 1. URL Structure
Nitter mirrors Twitter's URL layout:
| Target | Nitter path |
|--------|------------|
| Twitter List | `/i/lists/<listId>` |
| User Profile | `/<handle>` |
| Paginated | append `?cursor=<cursor>` |
### 2. Fetch Strategy: axios → Playwright fallback
Start with plain HTTP (`axios`). If nitter serves an anti-bot challenge page (detected by the string `"Verifying your request"` in the HTML), permanently switch to a headed Playwright/Chrome session for the rest of the run.
```ts
// challengeSolver.ts — detect challenge
const CHALLENGE_MARKER = 'Verifying your request';
export function isChallengePage(html: string): boolean {
return html.includes(CHALLENGE_MARKER);
}
// scrapeNitterList.ts — switch modes on detection
if (isChallengePage(html)) {
useBrowserMode = true;
const browserHtml = await fetchWithBrowser(url, debug);
return { html: browserHtml, url };
}
```
### 3. Browser Anti-bot Bypass
When challenge is detected, launch a real system Chrome (not headless) with automation signals suppressed:Details
- Skill name
twitter-x-scraping- Repository
- swyxio/skills
- Path
twitter-x-scraping/SKILL.md- License
- MIT
- Repo updated
- 16 November 2026
FAQ
What does the Twitter X Scraping skill do?
Use when scraping public Twitter/X timelines or lists through Nitter-style mirrors with axios and Playwright fallback, including anti-bot challenge handling and pagination limits.
How do I install the Twitter X Scraping skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add swyxio/skills --skill twitter-x-scraping. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Twitter X Scraping popular?
It lives in swyxio/skills, which has 201 GitHub stars and 12 forks.
Related skills
Accelevents API
201swyxio/skills
Use when reading or updating AI Engineer Europe speaker records through the Accelevents REST API, especially for full-record PUT updates, auth header quirks, and preserving existing speaker fields.
Accelevents Speaker Sync
201swyxio/skills
Use when website speaker, session, schedule, room, track, or headshot changes must be synchronized back to Accelevents for AI Engineer Europe 2026.
Agents SDK
201swyxio/skills
Build or debug stateful applications that specifically use Cloudflare's `agents` package, including Agent classes, routing, callable methods, synchronized state, scheduling, AIChatAgent, durable execution, or Agents SDK client hooks. Do not trigger for generic AI-agent design, the OpenAI Agents SDK, or ordinary Durable Object work that does not use the Cloudflare Agents SDK.
Web Scraper
150guia-matthieu/clawfu-skills
Extract structured data from websites. Use when: collecting competitor pricing; scraping product listings; extracting contact information; gathering research data; monitoring website changes
Statistical Distance
191jwynia/agent-skills
Transform clichéd story elements by pushing along the emotional vector toward statistical edges. Use when first instincts are too predictable, when elements feel generic, or when you need the core methodology for avoiding statistical-center defaults.
Data Visualization
223NeverSight/learn-skills.dev
Data visualization with chart selection, color theory, and annotation best practices. Covers chart types (bar, line, scatter, heatmap), axes rules, and storytelling with data. Use for: charts, graphs, dashboards, reports, presentations, infographics, data stories. Triggers: data visualization, chart, graph, data chart, bar chart, line chart, scatter plot, data viz, visualization, dashboard chart, infographic data, data presentation, chart design, plot, heatmap, pie chart alternative
Browse the most popular Claude skills, all data & analytics skills, or write your own with the skills generator.