polite-crawler MCP Server: Install & Setup

Install polite-crawler in Claude Code

Claude Code

claude mcp add polite-crawler -- npx -y polite-crawler-mcp

mcp.json (Claude Desktop, Cursor, VS Code, Windsurf)

{
  "mcpServers": {
    "polite-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "polite-crawler-mcp"
      ]
    }
  }
}

Frequently asked questions

What does the polite-crawler MCP server do?

A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.

Is the polite-crawler MCP server free?

Yes, most likely. Open source and no paid account or API key is mentioned in the listing. Check the provider for current pricing.

Does polite-crawler run locally or remotely?

Local. Runs on your machine (stdio). You install it with npx, uvx, Docker or from source.

How do I add polite-crawler to Claude Code?

Run: claude mcp add polite-crawler -- npx -y polite-crawler-mcp

More search & web scraping MCP servers

View all

Activitypub

17353

cameronrye/activitypub-mcp

Explore and interact with the Fediverse via ActivityPub: WebFinger discovery, timelines and instance exploration across Mastodon, Pleroma, Misskey and more.

LocalFreeTypeScript

M3 Memory

skynetcmd/m3-memory

Local-first, cloud-friendly memory — 100+ tools, 99.2% LongMemEval-S retrieval@8, hybrid search.

LocalFree

Books

863

mrfentmen/books-mcp

Search books and fetch volume details from the Google Books API. No key required.

LocalFree

Iconoteka

13513

iconoteka/iconoteka

Search 1,400 open-source icons in 7 weights and 2 styles, and get paste-ready SVG.

LocalFree

Glippy

871

jbobbink/glippy

GEO (Generative Engine Optimization) analysis: check any domain's AI-readiness for LLM search

LocalAccount needed

Everything File Search

elis132/everything-mcp

Lightning-fast Windows file search for AI agents via voidtools Everything's real-time NTFS index

LocalFree