Install AI Crawler Policy API in Claude Code
claude mcp add --transport http ai-crawler-policy-api https://api.lazy-mac.com/ai-crawler-policy/mcp{
"mcpServers": {
"ai-crawler-policy-api": {
"url": "https://api.lazy-mac.com/ai-crawler-policy/mcp"
}
}
}Frequently asked questions
What does the AI Crawler Policy API MCP server do?
Cloudflare Workers MCP server: ai-crawler-policy
Is the AI Crawler Policy API MCP server free?
The server is free, but you will likely need an account. The server itself is free, but it connects to a service that needs an account or API key, which may have paid tiers. Check the provider for current pricing.
Does AI Crawler Policy API run locally or remotely?
Remote. Hosted by the provider. You connect with a URL, nothing to install.
How do I add AI Crawler Policy API to Claude Code?
Run: claude mcp add --transport http ai-crawler-policy-api https://api.lazy-mac.com/ai-crawler-policy/mcp
More search & web scraping MCP servers
View allScrapeGraphAI
ScrapeGraphAI/Scrapegraph-ai
AI-powered web scraping library that creates scraping pipelines using natural language.- ScrapeGraphAI
Repomix
yamadashy/repomix
Pack local or remote codebases into AI-friendly files that LLMs and coding agents can read or search
Agent Skills Search
agentskills/agentskills
Search and discover Agent Skills from the skills.sh registry. Powered by HAPI MCP server.
screenpipe
screenpipe/screenpipe
Search your local screen recordings, audio transcripts, and computer activity from screenpipe.
XERJ
xerj-org/xerj
The search engine for AI agents: cited answers from your local code, docs, specs and logs.
Apify
apify/apify-mcp-server
Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡