Commoncrawl MCP Server: Install & Setup

Install Commoncrawl in Claude Code

Remote URL

Claude Code

claude mcp add --transport http pipeworx-io-commoncrawl https://gateway.pipeworx.io/commoncrawl/mcp

mcp.json (Claude Desktop, Cursor, VS Code, Windsurf)

{
  "mcpServers": {
    "pipeworx-io-commoncrawl": {
      "url": "https://gateway.pipeworx.io/commoncrawl/mcp"
    }
  }
}

npm

Claude Code

claude mcp add pipeworx-io-commoncrawl -- npx -y @pipeworx/mcp-commoncrawl

mcp.json (Claude Desktop, Cursor, VS Code, Windsurf)

{
  "mcpServers": {
    "pipeworx-io-commoncrawl": {
      "command": "npx",
      "args": [
        "-y",
        "@pipeworx/mcp-commoncrawl"
      ]
    }
  }
}

Frequently asked questions

What does the Commoncrawl MCP server do?

Common Crawl's public web archive — find every time a URL was crawled since 2008 and read the…

Is the Commoncrawl MCP server free?

Yes, most likely. Open source and no paid account or API key is mentioned in the listing. Check the provider for current pricing.

Does Commoncrawl run locally or remotely?

Local + Remote. Can run locally or be used as a hosted remote server.

How do I add Commoncrawl to Claude Code?

Run: claude mcp add --transport http pipeworx-io-commoncrawl https://gateway.pipeworx.io/commoncrawl/mcp

More search & web scraping MCP servers

View all

Commune

shanjai-raj/commune-mcp

Email and SMS MCP server for AI agents — 41 tools for inboxes, sending, search, and SMS.

LocalAccount needed

Coding Challenge Solutions

nikhil-bhat/cc-solutions-mcp

Search coding challenge solutions by keyword and optionally filter by programming language

LocalFree

conduitAI

akbaraziz/conduitai-bolt

Search, browse, and install MCP servers, agents, skills, and plugins from conduitAI.

LocalFree

CoderSwap

njlnaet/mcp-server

Build searchable AI research knowledge bases with hybrid vector search directly in Claude Desktop.

LocalFree

Congressional Documents

pipeworx-io/mcp-congressional-documents

Congressional Documents — full-text search and retrieval over the official record of Congress…

Local + RemoteFree

Codeintel

phuongddx/codeintel

Local-first SCIP code navigation and Zoekt search over your own indexed repositories

LocalFree