Benchmark Model

Benchmark a model served on MAX with the `max benchmark` command: measure throughput (tokens/sec), latency (TTFT, TPOT, inter-token latency), and GPU utilization by driving load against a running `max serve` endpoint. Use this whenever the user wants to benchmark, load-test, or measure the performance of a MAX model, get tokens-per-second / TTFT / TPOT numbers, run a concurrency or request-rate sweep, compare latency vs throughput, size a deployment, or produce benchmark JSON, even if they don't say "benchmark" by name. Also use when a `max benchmark` run fails to connect or reports zero/garba

213 stars 22 forksAI, LLMs & Agents Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add modular/skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add modular/skills --skill benchmark-model
Manual install
git clone --depth 1 https://github.com/modular/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/benchmark-model ~/.claude/skills/benchmark-model

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Benchmark a model on MAX

`max benchmark` measures a **running** model server. It's a load generator: it
sends inference requests to a live `max serve` endpoint, times them, and reports
throughput (tokens/sec) and latency (TTFT, TPOT, inter-token latency). A
benchmark combines two things: a server under test, and a workload that matches
the question you're asking.

Every run reports both throughput and latency, so there's no mode to select.
Decide what you want to learn first, then pick the workload that measures it.
Single-stream latency and peak throughput come from different workloads, so the
workload you choose *is* the measurement. Get that right and a clean number
falls out.

Use this skill when you want a performance number for a model on MAX:
tokens/sec, TTFT / TPOT, a concurrency or request-rate sweep, a
latency-vs-throughput tradeoff, or deployment sizing.

Don't use this skill when no server is running yet. `max benchmark` is a client,
so start a server with the `serve-model` skill first. To find out *where*
inference time goes at the kernel level, use `profile-model`. To check *whether
the output is correct*, treat it as a parity task (`import-model`, then
`debug-model`) rather than a benchmark.

This skill works anywhere MAX is installed (pip or pixi). Add `pixi run` in a
pixi project.

## References

The following table lists the reference files and when to read each one:

| File | Read when |
|------|-----------|
Read the full skill on GitHub

Details

Skill name
benchmark-model
Repository
modular/skills
Path
benchmark-model/SKILL.md
Repo updated
18 November 2026

FAQ

What does the Benchmark Model skill do?

Benchmark a model served on MAX with the `max benchmark` command: measure throughput (tokens/sec), latency (TTFT, TPOT, inter-token latency), and GPU utilization by driving load against a running `max serve` endpoint. Use this whenever the user wants to benchmark, load-test, or measure the performance of a MAX model, get tokens-per-second / TTFT / TPOT numbers, run a concurrency or request-rate sweep, compare latency vs throughput, size a deployment, or produce benchmark JSON, even if they don't say "benchmark" by name. Also use when a `max benchmark` run fails to connect or reports zero/garba

How do I install the Benchmark Model skill?

Claude Code plugin: /plugin marketplace add modular/skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Benchmark Model popular?

It lives in modular/skills, which has 213 GitHub stars and 22 forks.

Related skills

Debug Model

213

modular/skills

Debug a MAX model that loads, compiles, serves, and generates tokens but whose output disagrees with a reference implementation. Use whenever parity debugging stalls on scalar taps, the model returns gibberish or wrong greedy tokens, logit cosine is high but argmax differs, or generation is coherent then diverges. Applies during an architecture port, a quantization bring-up, a multi-GPU conversion, or after a MAX upgrade. Triggers on "parity failure", "silent corruption", "logits match but tokens diverge", "top-1 mismatch", "greedy divergence", and "model serves but generates garbage". Not for

AI, LLMs & Agents22

Eval Model

213

modular/skills

Measures the task accuracy of text models served by MAX using standard benchmarks such as GSM8K, MMLU, HellaSwag, ARC, AIME, GPQA, TruthfulQA, WinoGrande, and BABILong. Use when benchmarking a served model, comparing it with model-card or reference scores, verifying that a new MAX model produces correct answers, or running repeatable dataset evaluations against a MAX OpenAI-compatible endpoint.

AI, LLMs & Agents22

Import Model

213

modular/skills

Use when importing a new model architecture into MAX from a Hugging Face model ID. Triggers on: "import a model into MAX", "add model to MAX", "bring up <HF model> in MAX". Workflow: inspect the Hugging Face config and modeling code, then scaffold from a similar ModuleV3 architecture. Implement each layer as a max.nn Module that matches HF, serve, then verify against the Hugging Face reference. When the server runs but output is wrong (gibberish, greedy mismatch, coherent-then-diverges), load debug-model for the divergence hunt.

AI, LLMs & Agents22

Waxa Eval

423

mizchi/skills

Use when iterating on a skill's prompt with the waxa CLI (https://github.com/mizchi/skills/tree/main/tools/waxa) — authoring scenarios, choosing graders, interpreting unclear-points, advancing a ledger, and judging convergence. Encodes the four-stage iteration pattern observed in real iter loops (structural fix → grader breadth → surface-form coverage → residual unclear) and the scenario-design pitfalls (blank-slate executor's network limit, prompt expectation explicitness, regex coverage of Japanese/English surface forms). Meta-skill: do NOT auto-invoke for routine skill edits; fires only whe

AI, LLMs & Agents

Drawio MCP Diagramming

206

thomast1906/github-copilot-agent-skills

Create and edit diagrams using the Draw.io MCP server — any shape, any vendor. USE FOR: draw me a diagram, create an architecture diagram, add Azure/AWS/GCP/Cisco/Kubernetes icons to a diagram, convert Mermaid to draw.io, fix overlapping arrows, edit a .drawio file, network topology diagrams, CI/CD pipeline diagrams, auth flow diagrams. Supports XML, Mermaid, and CSV. Uses drawio/search_shapes to find any of 10,000+ shapes across all vendor and icon libraries. DO NOT USE FOR: Excalidraw output (use excalidraw-mcp-diagramming skill).

AI, LLMs & Agents54

Skill Finder

423

mizchi/skills

Use ONLY when the user explicitly asks to discover or evaluate a skill from OUTSIDE the curated catalog — e.g., "find a Stripe skill", "is there a skill for X?", "evaluate this candidate before adopting". Meta-skill, complementary to skill-selector: do NOT auto-invoke on routine apm-management, project-init, or catalog-resident requests. Cross-source survey across vetted registries (Anthropic official, claude-skill-registry, VoltAgent/awesome-agent-skills, ComposioHQ, Superpowers, GitHub topic) with a mandatory waxa-eval adoption gate.

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.