AIonDemandCluster

AI on Demand Cluster (aiod)

Spin up any HuggingFace model on a vast.ai GPU, serve it with vLLM, and drive
it from Claude Code through Claude Code Router (CCR) — all from one command.

Give it a HuggingFace link; aiod figures out how much VRAM the model needs, finds
the cheapest matching machine on vast.ai, shows you the live $/hr before you commit,
rents it, waits for the model to load, and writes your CCR config so you can run
ccr code against your own hosted model.

Why CCR sits in the middle: Claude Code speaks the Anthropic Messages API, while
vLLM serves the OpenAI API. CCR runs locally and translates between them, so the
remote box can stay a plain OpenAI-compatible server.

Prerequisites

- Python 3.10+
- A vast.ai account (sign up) + an API key from
  Account → API Keys (required)
- A HuggingFace token — only for gated/private models like Llama/Gemma (optional)
- Claude Code Router (classic CLI) installed locally:
Note: the project has a Desktop app (v3) that stores config under
~/Library/Application Support/.... aiod targets the classic CLI, whose
config lives at ~/.claude-code-router/config.json and uses ccr start/code/stop.

Install
Early release — install from source for now. PyPI (pipx install aiondemandcluster)
is wired up and coming shortly.

Then run aiod init to set up your keys.

Once on PyPI (pipx / uv / pip)

Dev setup (tests + lint)

Configure

Guided setup (recommended) — opens the right pages, validates each key as you paste
it, and writes .env for you:

Check an existing setup any time:

Or configure manually:

.env is gitignored — secrets never reach the public repo. Keys are read from (in
order) environment variables → project .env → global ~/.config/aiod/.env, so a
global install (pipx install / uv tool install) finds your keys from any…