Serving Llms on Instinct
Serves AI models on AMD Instinct GPU hardware using vLLM. Use this skill whenever the user wants to run, serve, deploy, start, host, or launch a language model on an AMD GPU, AMD Instinct, MI300X, MI325X, MI350X, or MI355X. Also use when the user mentions vLLM on ROCm, vLLM on AMD, serving on HBM, or asks how to get a model running on AMD data center hardware. Use when the user asks "run Qwen3", "serve DeepSeek", "start a vLLM endpoint", "get a model running on my AMD machine", or any similar phrasing. Handles the full flow: GPU detection, environment validation, vLLM configuration, launch, an
Install
/plugin marketplace add amd/skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add amd/skills --skill serving-llms-on-instinctgit clone --depth 1 https://github.com/amd/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/skills/serving-llms-on-instinct ~/.claude/skills/serving-llms-on-instinctPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Serving LLMs on AMD Instinct Get a vLLM endpoint running on AMD Instinct GPU hardware. ## Prerequisites - ROCm driver and `amd-smi` installed on the GPU host - Docker running and accessible (check with `docker ps`) - `/dev/kfd` and `/dev/dri` present on the GPU host - HuggingFace token in `HF_TOKEN` env var (required for gated models; not required for Qwen3 or Gemma). For gated models (Llama 3.2, Gemma, etc.), the HF token must belong to an account that has accepted the model's license at `huggingface.co/<model_id>`. A valid token without license acceptance will fail with an opaque "Engine core initialization failed" error. - For remote GPU: SSH key access configured (`ssh <user>@<host>` must work without a password prompt). If only password access is available, set up keys first: `ssh-copy-id <user>@<host>` ## Data files Read these files directly to get model and GPU configuration: - **`data/recipes_cache.json`** -- model configs synced from [vllm-project/recipes](https://github.com/vllm-project/recipes). Each entry under `models.<HF_ID>.recipe` contains the full recipe with `model.base_args`, `model.base_env`, `features.tool_calling.args`, `features.reasoning.args`, `hardware_overrides.amd.extra_args`, `hardware_overrides.amd.extra_env`. The top-level `docker_image` field has the latest resolved vLLM ROCm image. - **`data/gpu_overrides.json`** -- GPU-specific configuration. Contains
Details
- Skill name
serving-llms-on-instinct- Repository
- amd/skills
- Path
skills/serving-llms-on-instinct/SKILL.md- License
- MIT
- Repo updated
- 18 November 2026
FAQ
What does the Serving Llms on Instinct skill do?
Serves AI models on AMD Instinct GPU hardware using vLLM. Use this skill whenever the user wants to run, serve, deploy, start, host, or launch a language model on an AMD GPU, AMD Instinct, MI300X, MI325X, MI350X, or MI355X. Also use when the user mentions vLLM on ROCm, vLLM on AMD, serving on HBM, or asks how to get a model running on AMD data center hardware. Use when the user asks "run Qwen3", "serve DeepSeek", "start a vLLM endpoint", "get a model running on my AMD machine", or any similar phrasing. Handles the full flow: GPU detection, environment validation, vLLM configuration, launch, an
How do I install the Serving Llms on Instinct skill?
Claude Code plugin: /plugin marketplace add amd/skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Serving Llms on Instinct popular?
It lives in amd/skills, which has 500 GitHub stars and 47 forks.
Related skills
Apu Memory Tuner
500amd/skills
Inspects and tunes the shared-vs-dedicated memory split on AMD Ryzen APUs with unified memory (UMA) so larger LLMs and image-gen models fit on the iGPU, or so reserved GPU memory is returned to the CPU. Use when the user mentions Ryzen AI, Strix Halo / Strix Point / Krackan / Phoenix / Hawk Point, Ryzen AI Max, gfx1150 / gfx1151 / gfx1152, integrated Radeon, iGPU memory, UMA Frame Buffer Size, AMD Variable Graphics Memory, VGM, GTT, GART, TTM, pages_limit, amd-ttm, amd-debug-tools, "shared GPU memory", "dedicated GPU memory", carve-out, "not enough VRAM", "out of VRAM", "GPU OOM", llama.cpp on
Eaiapps Guided Install
500amd/skills
Guides a user through installing the AI Workbench (AIWB) onto a Kubernetes cluster end-to-end — preflight cluster readiness (access, StorageClass, GPU operator, required operators/CRDs), install the Helm charts in the correct order, and verify the deployment is healthy with plain-language failure triage. Use when a user wants to install, deploy, or stand up AIWB / the AI Workbench / the enterprise AI stack on a K8s cluster, or asks why their install failed partway. Use PROACTIVELY when a user is following helm/INSTALL.md by hand. Do NOT activate for onboarding models to AIM, general kubectl/he
Flydsl Kernel Authoring
500amd/skills
Comprehensive reference for authoring FlyDSL GPU kernels on AMD GPUs. Covers the layout algebra, tiled copy/MMA, buffer ops, loop-carried range loops, SharedAllocator (LDS), autotuning, and common patterns. Use when writing, reviewing, or understanding FlyDSL kernel code.
Tracelens Analysis Orchestrator
500amd/skills
Orchestrates modular PyTorch profiler trace analysis with TraceLens: generates perf reports, prepares category data, runs system-level and compute-kernel subagents in parallel, validates outputs, and writes a prioritized stakeholder report (analysis.md). Use when the user asks to follow the analysis orchestrator, run the agentic analysis workflow, run it from config <run.yaml>, analyze a trace, compare two traces, or mentions standalone or comparative TraceLens analysis.
Serving Llms on Epyc
500amd/skills
Serves an LLM on a supported AMD EPYC server CPU using vLLM with zentorch, in Docker, Podman, or conda. Use for "vLLM on CPU", "zentorch serving", or an EPYC CPU endpoint, including on a host that also has AMD Instinct GPUs. Detects the EPYC generation, validates the runtime, checks model support and RAM fit, sizes threads/KV/NUMA, confirms the plan, launches, and verifies the endpoint. Runs one instance on one socket and its memory. Reports and stops on failure; does not retry or debug. Use serving-llms-on-instinct when the endpoint should run on a GPU. Excludes multi-node, EPYC 4000, and pre
Awp
232awp-core/awp-skill
AWP (Agent Work Protocol) — the complete toolkit for agent mining on Base, Ethereum, Arbitrum, and BSC. Use this skill when the user explicitly mentions AWP, worknets, veAWP, awp-wallet, or AWP-specific operations. Handles: onboarding (wallet setup, registration, worknet joining), staking (deposit, withdraw, gasless relay via ERC-2612 permit), allocation (allocate/deallocate stake to agents on worknets), worknet management (register, pause, resume, cancel), agent binding (link agent wallet to owner), governance (proposals, voting), and querying (balances, positions, emissions, epochs, announce
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.