LLM Kv Cache Paged Attention

Use when the user asks about KV cache size, PagedAttention, vLLM OOM, gpu_memory_utilization, max_num_seqs, prefix caching, GQA/MLA memory or long-context OOM. Produces KV-per-token math, max concurrency and vLLM flags. Not for weight quantization (use llm-model-quantization) or local llama.cpp setups (use local-slm-inference).

45 stars 6 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add j4flmao/agent-skills --skill llm-kv-cache-paged-attention
Manual install
git clone --depth 1 https://github.com/j4flmao/agent-skills.git /tmp/agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/agent-skills/skills/ai/llm-engineering/kv-cache ~/.claude/skills/llm-kv-cache-paged-attention

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# KV Cache & PagedAttention

## Purpose
During decoding each new token attends to the keys and values of all previous tokens. Caching K and V avoids recomputing them, but the cache grows linearly with sequence length and batch size and usually becomes the binding constraint on concurrency. This skill gives the sizing math and the serving-engine levers (paging, prefix sharing, KV quantization, scheduling) to maximize throughput without OOM.

## Agent Protocol

### Trigger
"KV cache", "PagedAttention", "vLLM OOM", "gpu_memory_utilization", "max_num_seqs", "prefix caching", "long context memory", "KV cache quantization".

### Input Context
- Model architecture: layers, KV heads, head dim (from `config.json`), dtype.
- GPU count/VRAM, target context length, expected concurrent requests.
- Prompt shape: shared system prompts or few-shot prefixes?

### Output Artifact
KV budget calculation and serving configuration (vLLM flags or equivalent) with expected max concurrency.

### Response Format
```
Model: {name} L={layers} kv_heads={h} head_dim={d} dtype={bytes}
KV per token: 2*L*h*d*bytes = {KiB}
KV budget: {GiB} -> tokens in cache ≈ {n}
Max concurrency at {ctx} tokens ≈ {n}
Config: --max-model-len {..} --gpu-memory-utilization {..} --kv-cache-dtype {..} --max-num-seqs {..}
```

### Completion Criteria
- [ ] KV bytes/token computed from the real config (KV heads, not attention heads).
- [ ] Budget accounts for weights and activation/workspace memory.
Read the full skill on GitHub

Details

Skill name
llm-kv-cache-paged-attention
Path
skills/ai/llm-engineering/kv-cache/SKILL.md
License
MIT
Repo updated
17 November 2026

FAQ

What does the LLM Kv Cache Paged Attention skill do?

Use when the user asks about KV cache size, PagedAttention, vLLM OOM, gpu_memory_utilization, max_num_seqs, prefix caching, GQA/MLA memory or long-context OOM. Produces KV-per-token math, max concurrency and vLLM flags. Not for weight quantization (use llm-model-quantization) or local llama.cpp setups (use local-slm-inference).

How do I install the LLM Kv Cache Paged Attention skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add j4flmao/agent-skills --skill llm-kv-cache-paged-attention. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is LLM Kv Cache Paged Attention popular?

It lives in j4flmao/agent-skills, which has 45 GitHub stars and 6 forks.

Related skills

Agent Observability

45

j4flmao/agent-skills

Use when the user asks about agent observability, tracing agent reasoning, debugging an agent loop, OpenTelemetry GenAI spans for LLM calls, LLM cost tracking per agent, agent audit logs or hallucination/eval monitoring in production. Produces instrumentation config, trace schema, dashboards, alerts and cost attribution. Not for generic service tracing without LLMs (use devops-opentelemetry).

DevOps & Cloud

AI Agent Architectures

45

j4flmao/agent-skills

Use when the user asks to build an AI agent, design an agent that calls tools and keeps memory, write an agent system prompt or pick an agent framework (Claude Agent SDK, OpenAI Agents SDK, LangGraph). Produces an architecture brief covering loop, tools, memory, guardrails and evals, routing depth to child skills. Not for workflow patterns (use ai-agentic-workflows).

AI, LLMs & Agents

AI Agent Core Loop

45

j4flmao/agent-skills

Use when the user asks about the agent loop, ReAct vs plan-and-solve, re-planning, Reflexion, an agent stuck in a loop, max iterations or termination conditions. Produces a loop spec or patch with state, success predicate, caps and no-progress detection. Not for overall agent design (use ai-agent-architectures) or tool schemas (use tool-calling-principles).

AI, LLMs & Agents

LLM Model Quantization

45

j4flmao/agent-skills

Use when the user asks to quantize a model, GGUF, Q4_K_M, llama-quantize, GPTQ, AWQ, NF4, bitsandbytes, INT8/FP8 weights or a model too big for the GPU. Produces format choice per engine, quantization commands, memory estimate and an eval gate. Not for NVFP4/FP8 kernel-level tuning (use extreme-quantization-mastery) or QLoRA training (use llm-finetuning).

AI, LLMs & Agents

LLM Finetuning

45

j4flmao/agent-skills

Use when the user asks to fine-tune an open model with LoRA or QLoRA, PEFT, SFT, instruction tuning, TRL SFTTrainer, DPO, train on my data or merge an adapter. Produces a go/no-go decision, chat-format dataset spec, training config, eval plan and serving steps. Not for multi-node training (use distributed-training) or post-training quantization (use llm-model-quantization).

AI, LLMs & Agents

LLM RAG Retrieval Internals

45

j4flmao/agent-skills

Use when the user asks to build a RAG pipeline over documentation, tune HNSW (M, efSearch) or IVF-PQ (nprobe), hybrid dense + BM25 search, reciprocal rank fusion, cross-encoder reranking, Self-RAG or DSPy. Produces index choice and parameters, a hybrid+rerank pipeline and recall evals. Not for query rewriting and RAG prompts (use prompt-rag-query-design).

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.