Install Crawlerpeek in Claude Code
claude mcp add crawlerpeek -- npx -y crawlerpeek{
"mcpServers": {
"crawlerpeek": {
"command": "npx",
"args": [
"-y",
"crawlerpeek"
]
}
}
}Frequently asked questions
What does the Crawlerpeek MCP server do?
Check whether AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can access a URL. Read-only.
Is the Crawlerpeek MCP server free?
Yes, most likely. Open source and no paid account or API key is mentioned in the listing. Check the provider for current pricing.
Does Crawlerpeek run locally or remotely?
Local. Runs on your machine (stdio). You install it with npx, uvx, Docker or from source.
How do I add Crawlerpeek to Claude Code?
Run: claude mcp add crawlerpeek -- npx -y crawlerpeek
More search & web scraping MCP servers
View allElnora
Elnora-AI/elnora-mcp-server
AI-powered bioprotocol optimization — generate, search, and manage lab protocols via MCP
Codemunch Pro
BigJai/codemunch-pro
Code indexing MCP: 15 tools, 10 languages, hybrid search, call graphs, O(1) retrieval.
boutquin/mcp-server-email
Multi-account IMAP/SMTP email server with 22 tools for reading, sending, searching, and organizing.
Chronicle
Leviathan0x0/Chronicle-MCP
Local, private AI chat history search & memory: BM25 over your Cursor, Claude and ChatGPT chats.
Factcheck
carlisia/mcp-factcheck
An MCP server that validates content against MCP specification using semantic search and AI
Casemgr
mstang/casemgr-mcp
Shared workspace your AI agent writes to. 184 tools, persistent memory, semantic search.