AI LLM Inference
Operational patterns for LLM inference: latency budgeting, tail-latency control, caching, batching/scheduling, quantization/compression, parallelism, and reliable serving at scale. Emphasizes production-grade performance, cost control, and observability.
Install
/plugin marketplace add Freddy-CHEN-ux/00_claude_skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add Freddy-CHEN-ux/00_claude_skills --skill ai-llm-inferencegit clone --depth 1 https://github.com/Freddy-CHEN-ux/00_claude_skills.git /tmp/00_claude_skills
mkdir -p ~/.claude/skills
cp -r /tmp/00_claude_skills/skills/ai-llm-inference ~/.claude/skills/ai-llm-inferencePersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# LLMOps – Inference & Optimization – Production Skill Hub **Modern Best Practices (January 2026)**: - Treat inference as a **systems problem**: SLOs, tail latency, retries, overload, and cache strategy. - Use **continuous batching / smart scheduling** when serving many concurrent requests (Orca scheduling: https://www.usenix.org/conference/osdi22/presentation/yu). - Use **KV-cache aware serving** (PagedAttention/vLLM: https://arxiv.org/abs/2309.06180) and **efficient attention kernels** (FlashAttention: https://arxiv.org/abs/2205.14135). - Use **speculative decoding** when latency is critical and draft-model quality is acceptable (speculative decoding: https://arxiv.org/abs/2302.01318). - Quantize only with **measured** quality impact and rollback plan (quantization must be validated on your eval set). This skill provides **production-ready operational patterns** for optimizing LLM inference performance, cost, and reliability. It centralizes **decision rules**, **optimization strategies**, **configuration templates**, and **operational checklists** for inference workloads. No theory. No narrative. Only what Claude can execute. --- ## When to Use This Skill Claude should activate this skill whenever the user asks for: - Optimizing LLM inference latency or throughput - Choosing quantization strategies (FP8/FP4/INT8/INT4) - Configuring vLLM, TensorRT-LLM, or DeepSpeed inference - Scaling LLM inference across GPUs (tensor/pipeline parallelism)
Details
- Skill name
ai-llm-inference- Repository
- Freddy-CHEN-ux/00_claude_skills
- Path
skills/ai-llm-inference/SKILL.md- Repo updated
- 10 January 2026
FAQ
What does the AI LLM Inference skill do?
Operational patterns for LLM inference: latency budgeting, tail-latency control, caching, batching/scheduling, quantization/compression, parallelism, and reliable serving at scale. Emphasizes production-grade performance, cost control, and observability.
How do I install the AI LLM Inference skill?
Claude Code plugin: /plugin marketplace add Freddy-CHEN-ux/00_claude_skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is AI LLM Inference popular?
It lives in Freddy-CHEN-ux/00_claude_skills, which has 8 GitHub stars and 3 forks.
Related skills
Add Integration
8Freddy-CHEN-ux/00_claude_skills
Build a new API integration for Nexus. Load when user mentions "add integration", "new integration", "integrate with", "connect to [service]", or "build [service] integration". Interactive workflow that discovers API endpoints, plans the integration, and creates a project for implementation.
AI Agents
8Freddy-CHEN-ux/00_claude_skills
Production-grade AI agent patterns with MCP integration, agentic RAG, handoff orchestration, multi-layer guardrails, and observability (modern best practices)
AI LLM
8Freddy-CHEN-ux/00_claude_skills
Production LLM engineering skill. Covers strategy selection (prompting vs RAG vs fine-tuning), dataset design, PEFT/LoRA, evaluation workflows, deployment handoff to inference serving, and lifecycle operations with cost/safety controls.
AI Ml Data Science
8Freddy-CHEN-ux/00_claude_skills
End-to-end data science patterns (modern best practices): problem framing -> data -> EDA -> feature engineering (with feature stores) -> modelling -> evaluation -> reporting, plus SQL transformation (SQLMesh). Emphasizes MLOps integration, drift monitoring, and production-ready workflows.
AI Ml Timeseries
8Freddy-CHEN-ux/00_claude_skills
Operational patterns, templates, and decision rules for time series forecasting (modern best practices): tree-based methods (LightGBM), deep learning (Transformers, RNNs), future-guided learning, temporal validation, feature engineering, generative TS (Chronos), and production deployment. Emphasizes explainability, long-term dependency handling, and adaptive forecasting.
AI Mlops
8Freddy-CHEN-ux/00_claude_skills
Complete MLOps skill covering production ML lifecycle and security. Includes data ingestion, model deployment, drift detection, monitoring, plus ML security (prompt injection, jailbreak defense, RAG security, privacy, governance). Modern automation-first patterns with multi-layered defenses.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.