Crw Self Host

Stand up your own fastCRW API server — single binary, Docker, or docker-compose with a bundled search-backend sidecar. Use when the user wants to run crw locally or on their own infra, configure renderers/proxies/ auth/LLM extraction, or understand the embedded vs proxy MCP modes.

1,268 stars 102 forksDevOps & Cloud Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add fastcrw/crw

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add fastcrw/crw --skill crw-self-host
Manual install
git clone --depth 1 https://github.com/fastcrw/crw.git /tmp/crw
mkdir -p ~/.claude/skills
cp -r /tmp/crw/skills/crw-self-host ~/.claude/skills/crw-self-host

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# crw-self-host — Stand up your own crw

One static binary, one config file. No Redis, no Postgres, no Node.js runtime in
the request path. Self-host free under AGPL-3.0, or point at `api.fastcrw.com`
for managed.

## When to use

- You want full data residency (URLs and queries never leave your infra).
- You want to run recurring crawls/audits at VPS cost, not per-page credits.
- You need to configure proxy pools, custom renderers, or LLM extraction.
- Step 0 before any self-hosted crw workflow.

## Install paths

Choose one. All install paths run the same Rust binary.

### MCP server (`crw-mcp`) — recommended for AI agent use

```bash
npx crw-mcp                           # zero install (npm; embedded engine, ~14 MB RAM)
brew install fastcrw/crw/crw-mcp           # Homebrew
cargo install crw-mcp                 # Cargo (~17 MB, full embedded)
docker run -i ghcr.io/fastcrw/crw crw-mcp  # Docker
pip install crw                       # Python SDK (auto-downloads binary on first use)
```

**Lean build** (~4.2 MB, no headless browser engine — proxy/cloud-only mode):
```bash
cargo build --profile release-small --no-default-features -p crw-mcp
```

### CLI (`crw`) — scrape from the terminal
Read the full skill on GitHub

Details

Skill name
crw-self-host
Repository
fastcrw/crw
Path
skills/crw-self-host/SKILL.md
License
AGPL-3.0
Repo updated
18 November 2026

FAQ

What does the Crw Self Host skill do?

Stand up your own fastCRW API server — single binary, Docker, or docker-compose with a bundled search-backend sidecar. Use when the user wants to run crw locally or on their own infra, configure renderers/proxies/ auth/LLM extraction, or understand the embedded vs proxy MCP modes.

How do I install the Crw Self Host skill?

Claude Code plugin: /plugin marketplace add fastcrw/crw. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Crw Self Host popular?

It lives in fastcrw/crw, which has 1,268 GitHub stars and 102 forks, and is included in 1 curated skill list.

Related skills

Crw

1.3k

fastcrw/crw

Scrape, crawl, map, and search the web using fastCRW's native /v1 API. Use when the user needs web page content, site-wide extraction, URL discovery, or web search results. Single binary, 14 MB RAM; /v2 exists separately for Firecrawl migration.

Writing & ContentIn 1 list102

Crw

1.3k

fastcrw/crw

Scrape, crawl, map, search, parse, and extract web data with fastCRW — the open-source, self-hostable Firecrawl alternative (single Rust binary, ~14 MB RAM, Firecrawl-compatible /v1 + /v2 API). Use whenever the user needs page content, site-wide extraction, URL discovery, web search, PDF parsing, structured JSON from pages, or change tracking. Also use when the user mentions Firecrawl, Tavily, Crawl4AI, or "scrape/crawl/map/fetch/get the page/read this site/search the web" — crw is a drop-in for the Firecrawl SDKs.

Data & AnalyticsIn 1 list102

Crw Best Practices

1.3k

fastcrw/crw

Reference skill for building production-ready crw integrations. Covers verb selection, call surfaces (CLI/MCP/REST), post-filtering strategies, context-window hygiene, Hybrid RAG patterns, common pitfalls, and crw-specific operational considerations (search backend limits, renderer pool, proxy rotation). Load this when writing application code that embeds crw, designing a multi-step agent workflow, or debugging an integration that isn't behaving as expected.

AI, LLMs & AgentsIn 1 list102

Bash Defensive Patterns

2k

rmyndharis/antigravity-skills

Master defensive Bash programming techniques for production-grade scripts. Use when writing robust shell scripts, CI/CD pipelines, or system utilities requiring fault tolerance and safety.

DevOps & Cloud295

Pup

1k

DataDog/pup

Datadog API CLI with 49 command groups, 300+ subcommands. Skills and domain agents for monitoring, logs, APM, security, and infrastructure.

DevOps & Cloud139

Cloud Architect

2k

rmyndharis/antigravity-skills

Expert cloud architect specializing in AWS/Azure/GCP multi-cloud infrastructure design, advanced IaC (Terraform/OpenTofu/CDK), FinOps cost optimization, and modern architectural patterns. Masters serverless, microservices, security, compliance, and disaster recovery. Use for cloud architecture, cost optimization, migration planning, or multi-cloud strategies.

DevOps & Cloud295

Browse the most popular Claude skills, all devops & cloud skills, or write your own with the skills generator.