Serving Llms on Instinct

Serves AI models on AMD Instinct GPU hardware using vLLM. Use this skill whenever the user wants to run, serve, deploy, start, host, or launch a language model on an AMD GPU, AMD Instinct, MI300X, MI325X, MI350X, or MI355X. Also use when the user mentions vLLM on ROCm, vLLM on AMD, serving on HBM, or asks how to get a model running on AMD data center hardware. Use when the user asks "run Qwen3", "serve DeepSeek", "start a vLLM endpoint", "get a model running on my AMD machine", or any similar phrasing. Handles the full flow: GPU detection, environment validation, vLLM configuration, launch, an

500 stars 47 forksAI, LLMs & Agents Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add amd/skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add amd/skills --skill serving-llms-on-instinct
Manual install
git clone --depth 1 https://github.com/amd/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/skills/serving-llms-on-instinct ~/.claude/skills/serving-llms-on-instinct

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Serving LLMs on AMD Instinct

Get a vLLM endpoint running on AMD Instinct GPU hardware.

## Prerequisites

- ROCm driver and `amd-smi` installed on the GPU host
- Docker running and accessible (check with `docker ps`)
- `/dev/kfd` and `/dev/dri` present on the GPU host
- HuggingFace token in `HF_TOKEN` env var (required for gated models; not
  required for Qwen3 or Gemma). For gated models (Llama 3.2, Gemma, etc.),
  the HF token must belong to an account that has accepted the model's license
  at `huggingface.co/<model_id>`. A valid token without license acceptance will
  fail with an opaque "Engine core initialization failed" error.
- For remote GPU: SSH key access configured (`ssh <user>@<host>` must work
  without a password prompt). If only password access is available, set up
  keys first: `ssh-copy-id <user>@<host>`

## Data files

Read these files directly to get model and GPU configuration:

- **`data/recipes_cache.json`** -- model configs synced from
  [vllm-project/recipes](https://github.com/vllm-project/recipes). Each entry
  under `models.<HF_ID>.recipe` contains the full recipe with `model.base_args`,
  `model.base_env`, `features.tool_calling.args`, `features.reasoning.args`,
  `hardware_overrides.amd.extra_args`, `hardware_overrides.amd.extra_env`.
  The top-level `docker_image` field has the latest resolved vLLM ROCm image.

- **`data/gpu_overrides.json`** -- GPU-specific configuration. Contains
Read the full skill on GitHub

Details

Skill name
serving-llms-on-instinct
Repository
amd/skills
Path
skills/serving-llms-on-instinct/SKILL.md
License
MIT
Repo updated
18 November 2026

FAQ

What does the Serving Llms on Instinct skill do?

Serves AI models on AMD Instinct GPU hardware using vLLM. Use this skill whenever the user wants to run, serve, deploy, start, host, or launch a language model on an AMD GPU, AMD Instinct, MI300X, MI325X, MI350X, or MI355X. Also use when the user mentions vLLM on ROCm, vLLM on AMD, serving on HBM, or asks how to get a model running on AMD data center hardware. Use when the user asks "run Qwen3", "serve DeepSeek", "start a vLLM endpoint", "get a model running on my AMD machine", or any similar phrasing. Handles the full flow: GPU detection, environment validation, vLLM configuration, launch, an

How do I install the Serving Llms on Instinct skill?

Claude Code plugin: /plugin marketplace add amd/skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Serving Llms on Instinct popular?

It lives in amd/skills, which has 500 GitHub stars and 47 forks.

Related skills

Apu Memory Tuner

500

amd/skills

Inspects and tunes the shared-vs-dedicated memory split on AMD Ryzen APUs with unified memory (UMA) so larger LLMs and image-gen models fit on the iGPU, or so reserved GPU memory is returned to the CPU. Use when the user mentions Ryzen AI, Strix Halo / Strix Point / Krackan / Phoenix / Hawk Point, Ryzen AI Max, gfx1150 / gfx1151 / gfx1152, integrated Radeon, iGPU memory, UMA Frame Buffer Size, AMD Variable Graphics Memory, VGM, GTT, GART, TTM, pages_limit, amd-ttm, amd-debug-tools, "shared GPU memory", "dedicated GPU memory", carve-out, "not enough VRAM", "out of VRAM", "GPU OOM", llama.cpp on

Productivity & Notes47

Eaiapps Guided Install

500

amd/skills

Guides a user through installing the AI Workbench (AIWB) onto a Kubernetes cluster end-to-end — preflight cluster readiness (access, StorageClass, GPU operator, required operators/CRDs), install the Helm charts in the correct order, and verify the deployment is healthy with plain-language failure triage. Use when a user wants to install, deploy, or stand up AIWB / the AI Workbench / the enterprise AI stack on a K8s cluster, or asks why their install failed partway. Use PROACTIVELY when a user is following helm/INSTALL.md by hand. Do NOT activate for onboarding models to AIM, general kubectl/he

DevOps & Cloud47

Flydsl Kernel Authoring

500

amd/skills

Comprehensive reference for authoring FlyDSL GPU kernels on AMD GPUs. Covers the layout algebra, tiled copy/MMA, buffer ops, loop-carried range loops, SharedAllocator (LDS), autotuning, and common patterns. Use when writing, reviewing, or understanding FlyDSL kernel code.

Writing & Content47

Tracelens Analysis Orchestrator

500

amd/skills

Orchestrates modular PyTorch profiler trace analysis with TraceLens: generates perf reports, prepares category data, runs system-level and compute-kernel subagents in parallel, validates outputs, and writes a prioritized stakeholder report (analysis.md). Use when the user asks to follow the analysis orchestrator, run the agentic analysis workflow, run it from config <run.yaml>, analyze a trace, compare two traces, or mentions standalone or comparative TraceLens analysis.

AI, LLMs & Agents47

Serving Llms on Epyc

500

amd/skills

Serves an LLM on a supported AMD EPYC server CPU using vLLM with zentorch, in Docker, Podman, or conda. Use for "vLLM on CPU", "zentorch serving", or an EPYC CPU endpoint, including on a host that also has AMD Instinct GPUs. Detects the EPYC generation, validates the runtime, checks model support and RAM fit, sizes threads/KV/NUMA, confirms the plan, launches, and verifies the endpoint. Runs one instance on one socket and its memory. Reports and stops on failure; does not retry or debug. Use serving-llms-on-instinct when the endpoint should run on a GPU. Excludes multi-node, EPYC 4000, and pre

AI, LLMs & Agents47

Awp

232

awp-core/awp-skill

AWP (Agent Work Protocol) — the complete toolkit for agent mining on Base, Ethereum, Arbitrum, and BSC. Use this skill when the user explicitly mentions AWP, worknets, veAWP, awp-wallet, or AWP-specific operations. Handles: onboarding (wallet setup, registration, worknet joining), staking (deposit, withdraw, gasless relay via ERC-2612 permit), allocation (allocate/deallocate stake to agents on worknets), worknet management (register, pause, resume, cancel), agent binding (link agent wallet to owner), governance (proposals, voting), and querying (balances, positions, emissions, epochs, announce

AI, LLMs & Agents51

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.