LocalFree

Crawlerpeek MCP server

crawlerpeek/crawlerpeek

Check whether AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can access a URL. Read-only.

3 GitHub stars1,435 npm downloads / month

Install Crawlerpeek in Claude Code

Claude Code
claude mcp add crawlerpeek -- npx -y crawlerpeek
mcp.json (Claude Desktop, Cursor, VS Code, Windsurf)
{
  "mcpServers": {
    "crawlerpeek": {
      "command": "npx",
      "args": [
        "-y",
        "crawlerpeek"
      ]
    }
  }
}

Frequently asked questions

What does the Crawlerpeek MCP server do?

Check whether AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can access a URL. Read-only.

Is the Crawlerpeek MCP server free?

Yes, most likely. Open source and no paid account or API key is mentioned in the listing. Check the provider for current pricing.

Does Crawlerpeek run locally or remotely?

Local. Runs on your machine (stdio). You install it with npx, uvx, Docker or from source.

How do I add Crawlerpeek to Claude Code?

Run: claude mcp add crawlerpeek -- npx -y crawlerpeek

More search & web scraping MCP servers

View all