RemoteFree

RobotsGate MCP server

dev.workers.mike-tusa.robotsgate/robotsgate

Generate, validate and check robots.txt rules for AI crawlers. Free; optional Pro key.

Install RobotsGate in Claude Code

Claude Code
claude mcp add --transport http robotsgate https://robotsgate.mike-tusa.workers.dev/mcp
mcp.json (Claude Desktop, Cursor, VS Code, Windsurf)
{
  "mcpServers": {
    "robotsgate": {
      "url": "https://robotsgate.mike-tusa.workers.dev/mcp"
    }
  }
}

Frequently asked questions

What does the RobotsGate MCP server do?

Generate, validate and check robots.txt rules for AI crawlers. Free; optional Pro key.

Is the RobotsGate MCP server free?

Yes, most likely. Open source and no paid account or API key is mentioned in the listing. Check the provider for current pricing.

Does RobotsGate run locally or remotely?

Remote. Hosted by the provider. You connect with a URL, nothing to install.

How do I add RobotsGate to Claude Code?

Run: claude mcp add --transport http robotsgate https://robotsgate.mike-tusa.workers.dev/mcp

More search & web scraping MCP servers

View all