Serve Model

Serve a model with MAX's `max serve` command: set up the environment (pixi or uv with the max-nightly conda channel / nightly wheel index), point the server at a Hugging Face repo or local checkpoint, target a custom architecture with `--custom-architectures`, and pick the right serve flags for the model. Use this whenever the user wants to run, launch, start, or host a model on MAX, bring up an OpenAI-compatible endpoint, serve a custom/ported architecture, debug a `max serve` startup failure, or figure out which serve flags (devices, quantization-encoding, max-length, task, trust-remote-code

213 stars 22 forksAI, LLMs & Agents Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add modular/skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add modular/skills --skill serve-model
Manual install
git clone --depth 1 https://github.com/modular/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/serve-model ~/.claude/skills/serve-model

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Serve a model with MAX

`max serve` launches an OpenAI-compatible HTTP server for a model. It handles
tokenization, batching, KV cache, and the whole serving stack. You point it at
a checkpoint and, if the model isn't built into MAX, at a custom architecture
package. This skill takes you from "no environment" to "server answering
requests," and helps you choose flags that fit the specific model instead of
guessing.

The guiding principle: **start from the smallest command that could work, then
add flags only when the model or the hardware forces you to**. MAX auto-detects
most things (dtype, sequence length, device defaults). Over-specifying flags is
the most common way people turn a working serve into a broken one.

**Use this skill when** you want to run, launch, or host a model on MAX: bring
up an OpenAI-compatible endpoint, serve a built-in or a custom/ported
architecture, or debug a `max serve` startup failure.

**Do not use this skill when** the model isn't implemented in MAX yet (no
working `arch.py`, graph, and weights). That's a bring-up task: use
`import-model` to port the architecture, and `debug-model` if it serves but the
output is wrong. This skill runs an existing model; it doesn't author one.

## References

| File                                                           | Read when                                                                                                  |
Read the full skill on GitHub

Details

Skill name
serve-model
Repository
modular/skills
Path
serve-model/SKILL.md
Repo updated
18 November 2026

FAQ

What does the Serve Model skill do?

Serve a model with MAX's `max serve` command: set up the environment (pixi or uv with the max-nightly conda channel / nightly wheel index), point the server at a Hugging Face repo or local checkpoint, target a custom architecture with `--custom-architectures`, and pick the right serve flags for the model. Use this whenever the user wants to run, launch, start, or host a model on MAX, bring up an OpenAI-compatible endpoint, serve a custom/ported architecture, debug a `max serve` startup failure, or figure out which serve flags (devices, quantization-encoding, max-length, task, trust-remote-code

How do I install the Serve Model skill?

Claude Code plugin: /plugin marketplace add modular/skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Serve Model popular?

It lives in modular/skills, which has 213 GitHub stars and 22 forks.

Related skills

Benchmark Model

213

modular/skills

Benchmark a model served on MAX with the `max benchmark` command: measure throughput (tokens/sec), latency (TTFT, TPOT, inter-token latency), and GPU utilization by driving load against a running `max serve` endpoint. Use this whenever the user wants to benchmark, load-test, or measure the performance of a MAX model, get tokens-per-second / TTFT / TPOT numbers, run a concurrency or request-rate sweep, compare latency vs throughput, size a deployment, or produce benchmark JSON, even if they don't say "benchmark" by name. Also use when a `max benchmark` run fails to connect or reports zero/garba

AI, LLMs & Agents22

Debug Model

213

modular/skills

Debug a MAX model that loads, compiles, serves, and generates tokens but whose output disagrees with a reference implementation. Use whenever parity debugging stalls on scalar taps, the model returns gibberish or wrong greedy tokens, logit cosine is high but argmax differs, or generation is coherent then diverges. Applies during an architecture port, a quantization bring-up, a multi-GPU conversion, or after a MAX upgrade. Triggers on "parity failure", "silent corruption", "logits match but tokens diverge", "top-1 mismatch", "greedy divergence", and "model serves but generates garbage". Not for

AI, LLMs & Agents22

Eval Model

213

modular/skills

Measures the task accuracy of text models served by MAX using standard benchmarks such as GSM8K, MMLU, HellaSwag, ARC, AIME, GPQA, TruthfulQA, WinoGrande, and BABILong. Use when benchmarking a served model, comparing it with model-card or reference scores, verifying that a new MAX model produces correct answers, or running repeatable dataset evaluations against a MAX OpenAI-compatible endpoint.

AI, LLMs & Agents22

Slm Governance

278

qualixar/superlocalmemory

Governed-workspace behavior for SuperLocalMemory. Covers roles (admin/member/viewer) and company mode, who administers a company-mode workspace (a signed-in admin, or `slm team` on the SLM computer), `slm token show` in strict mode, what remote web apps can never see or change, retention and lifecycle settings, the audit trail, GDPR export and erasure, and how agents must behave when operating under workspace governance. The audit and retention tools need the power MCP profile. Agents must never bypass governance controls.

AI, LLMs & AgentsIn 1 list36

Profile Model

213

modular/skills

Profile a model running on MAX to find where it spends time and whether the GPU is saturated. Use when the user asks to "profile my model," "where is my model spending time," "why is inference slow," "is my GPU being utilized," "how much GPU am I using," "get a kernel breakdown," "capture an nsys/rocprof/ncu trace of max serve," or wants to measure MAX inference performance. Works for any model MAX can run — built-in architectures and custom ones loaded with --custom-architectures — from a pip or pixi install (max generate, max serve, or a Python script) on NVIDIA or AMD GPUs. Decide cheapest-

AI, LLMs & Agents22

Slm Mesh

278

qualixar/superlocalmemory

Cross-session peer coordination via the SLM mesh network. Lets multiple AI agent sessions on the same machine, and web apps the owner has allowed (ChatGPT, Claude on the web, Grok Bot, Muse, Composio), discover each other, send messages, wait for replies with mesh_wait, share lightweight state, and lock files to avoid conflicts. Covers the at-least-once delivery rule for web apps (pass ack_ids back as ack) and the two permissions a web app needs. Available in the default tool set and in the full, power and mesh MCP profiles. Only `slm mesh status` and `slm mesh peers` exist on the command line

AI, LLMs & AgentsIn 1 list36

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.