Ollama Herd

Ollama Herd

Ollama Herd is an open-source smart AI router that turns the machines you already own (Apple Silicon Macs, Linux servers, and Windows PCs, with or without NVIDIA GPUs) into one endpoint. It routes LLMs, embeddings, image generation, speech-to-text, and vision with 8-signal scoring, mDNS auto-discovery, and a real-time dashboard. Apple Silicon Macs get extras: an MLX backend, native image generation, and speech-to-text.

Works with Claude Code (Anthropic Messages API), Codex (OpenAI Responses API), Open WebUI, and any OpenAI- or Ollama-compatible client, all on one URL.

ollamaherd.com: docs, guides, and the dashboard walkthrough.

Why Ollama Herd?

- Every machine you own, one endpoint: Macs, Linux servers, and Windows PCs pooled into one fleet
- Agents stop queuing behind one Ollama: each request goes to the best machine for that model right now
- Claude Code and Codex on your own hardware: model ids auto-route to the best model you have loaded, no model map needed
- No config files, no Docker, no Kubernetes: two commands, mDNS auto-discovery
- Not just LLMs: embeddings, image generation, and speech-to-text route too (image generation and speech-to-text extras on Apple Silicon)
- Start on one machine: run both commands on it, and add machines when you have them

Quick Start

Or with Homebrew (macOS/Linux):
Without the brew trust line, Homebrew 6.0+ stops with *"Refusing to load formula … from untrusted tap"*. Older Homebrew doesn't need it and ignores it harmlessly. The install builds every dependency from source (Homebrew runs pip --no-binary :all:), including a Rust compile of pydantic-core — budget ~25 minutes, not the couple of minutes a bottled formula takes.

On your router machine:

On each device running Ollama:

One machine? Run both commands on…