Webclaw
Web extraction for AI agents. Scrape, crawl, map, batch, extract, summarize, diff, brand, search, and 28 site-specific extractors turn any URL into clean Markdown, text, or JSON. Runs locally with no key via the webclaw MCP server; set an optional WEBCLAW_API_KEY to handle bot-protected and JavaScript-rendered pages. Use when web_fetch returns blocked or empty content, or you need structured, LLM-ready extraction.
Install
npx skills add NeverSight/learn-skills.dev --skill webclawgit clone --depth 1 https://github.com/NeverSight/learn-skills.dev.git /tmp/learn-skills.dev
mkdir -p ~/.claude/skills
cp -r /tmp/learn-skills.dev/data/skills-md/0xmassi/webclaw-skill/webclaw ~/.claude/skills/webclawPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# webclaw Web extraction for AI agents, powered by a local Rust engine. It turns any URL into clean Markdown, text, or JSON through the webclaw MCP server, and it works with no API key. ## Install ``` npx create-webclaw ``` This downloads the `webclaw-mcp` binary and adds it to your agent's MCP config (Claude Code, Cursor, Windsurf, Codex, Antigravity, and more). Restart the agent to load the tools below. - **No key needed.** Extraction runs on your machine: free, private, and unlimited for the common case. Static sites, docs, blogs, server-rendered pages, and product pages all work locally. - **Optional upgrade.** Set `WEBCLAW_API_KEY` (from https://webclaw.io) and webclaw escalates to the hosted engine for the pages local extraction can't finish: bot-protected sites and JavaScript-rendered SPAs. Without a key, those pages return a clear message that tells you how to unlock them. ## When to use this skill - You need reliable web content with clean, structured output. - `web_fetch` returns empty, truncated, or blocked content. - You need structured data (pricing tables, product specs, contact info) as JSON. - You need to crawl a whole site or discover every URL. - You need LLM-optimized content, cleaner than raw markdown. - You need a site-specific extractor (GitHub, Reddit, YouTube, npm, PyPI, Amazon). - You need to summarize, diff, or search the web.
Details
- Skill name
webclaw- Repository
- NeverSight/learn-skills.dev
- Path
data/skills-md/0xmassi/webclaw-skill/webclaw/SKILL.md- Repo updated
- 18 November 2026
FAQ
What does the Webclaw skill do?
Web extraction for AI agents. Scrape, crawl, map, batch, extract, summarize, diff, brand, search, and 28 site-specific extractors turn any URL into clean Markdown, text, or JSON. Runs locally with no key via the webclaw MCP server; set an optional WEBCLAW_API_KEY to handle bot-protected and JavaScript-rendered pages. Use when web_fetch returns blocked or empty content, or you need structured, LLM-ready extraction.
How do I install the Webclaw skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add NeverSight/learn-skills.dev --skill webclaw. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Webclaw popular?
It lives in NeverSight/learn-skills.dev, which has 223 GitHub stars and 42 forks.
Related skills
Council
223NeverSight/learn-skills.dev
Convene the Council of High Intelligence — multi-persona deliberation with historical thinkers for deeper analysis of complex problems.
Megaeth Developer
223NeverSight/learn-skills.dev
End-to-end MegaETH development playbook (Feb 2026). Covers wallet operations, token swaps (Kyber Network), eth_sendRawTransactionSync (EIP-7966) for instant receipts, JSON-RPC batching, real-time mini-block subscriptions, storage-aware contract patterns (Solady RedBlackTreeLib), MegaEVM gas model, WebSocket keepalive, bridging from Ethereum, and debugging with mega-evme. Use when building on MegaETH, managing wallets, sending transactions, or deploying contracts.
Openclaw Stock Skill
223NeverSight/learn-skills.dev
使用 data.diemeng.chat 提供的接口查询股票日线、分钟线、财务指标等数据,支持 A 股等市场。
Yellow Best Practices
223NeverSight/learn-skills.dev
Yellow Network and Nitrolite (ERC-7824) development best practices for building state channel applications. Use when building apps with Yellow SDK, implementing state channels, connecting to ClearNodes, managing off-chain transactions, or working with application sessions.
Unfold Admin
223NeverSight/learn-skills.dev
Django Unfold admin theme - build, configure, and enhance modern Django admin interfaces with Unfold. Use when working with: (1) Django admin UI customisation or theming, (2) Unfold ModelAdmin, inlines, actions, filters, widgets, or decorators, (3) Admin dashboard components and KPI cards, (4) Sidebar navigation, tabs, or conditional fields, (5) Any mention of 'unfold', 'django-unfold', or 'unfold admin'. Covers the full Unfold feature set: site configuration, actions system, display decorators, filter types, widget overrides, inline variants, dashboard components, datasets, sections, theming,
Yolo
223NeverSight/learn-skills.dev
Browser automation skill for Lovable deployments. Activates when: - yolo_mode: on in CLAUDE.md - Running /deploy-edge or /apply-migration commands - After git push when auto_deploy: on (automatic detection) - Any mention of "yolo mode", "automate Lovable", "browser automation" Automatically navigates to Lovable.dev and submits deployment prompts. Runs verification tests based on configuration. Auto-deploys after git push when enabled.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.