AI LLM Inference

Operational patterns for LLM inference: latency budgeting, tail-latency control, caching, batching/scheduling, quantization/compression, parallelism, and reliable serving at scale. Emphasizes production-grade performance, cost control, and observability.

8 stars 3 forksAI, LLMs & Agents Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add Freddy-CHEN-ux/00_claude_skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add Freddy-CHEN-ux/00_claude_skills --skill ai-llm-inference
Manual install
git clone --depth 1 https://github.com/Freddy-CHEN-ux/00_claude_skills.git /tmp/00_claude_skills
mkdir -p ~/.claude/skills
cp -r /tmp/00_claude_skills/skills/ai-llm-inference ~/.claude/skills/ai-llm-inference

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# LLMOps – Inference & Optimization – Production Skill Hub

**Modern Best Practices (January 2026)**:

- Treat inference as a **systems problem**: SLOs, tail latency, retries, overload, and cache strategy.
- Use **continuous batching / smart scheduling** when serving many concurrent requests (Orca scheduling: https://www.usenix.org/conference/osdi22/presentation/yu).
- Use **KV-cache aware serving** (PagedAttention/vLLM: https://arxiv.org/abs/2309.06180) and **efficient attention kernels** (FlashAttention: https://arxiv.org/abs/2205.14135).
- Use **speculative decoding** when latency is critical and draft-model quality is acceptable (speculative decoding: https://arxiv.org/abs/2302.01318).
- Quantize only with **measured** quality impact and rollback plan (quantization must be validated on your eval set).

This skill provides **production-ready operational patterns** for optimizing LLM inference performance, cost, and reliability. It centralizes **decision rules**, **optimization strategies**, **configuration templates**, and **operational checklists** for inference workloads.

No theory. No narrative. Only what Claude can execute.

---

## When to Use This Skill

Claude should activate this skill whenever the user asks for:

- Optimizing LLM inference latency or throughput
- Choosing quantization strategies (FP8/FP4/INT8/INT4)
- Configuring vLLM, TensorRT-LLM, or DeepSpeed inference
- Scaling LLM inference across GPUs (tensor/pipeline parallelism)
Read the full skill on GitHub

Details

Skill name
ai-llm-inference
Path
skills/ai-llm-inference/SKILL.md
Repo updated
10 January 2026

FAQ

What does the AI LLM Inference skill do?

Operational patterns for LLM inference: latency budgeting, tail-latency control, caching, batching/scheduling, quantization/compression, parallelism, and reliable serving at scale. Emphasizes production-grade performance, cost control, and observability.

How do I install the AI LLM Inference skill?

Claude Code plugin: /plugin marketplace add Freddy-CHEN-ux/00_claude_skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is AI LLM Inference popular?

It lives in Freddy-CHEN-ux/00_claude_skills, which has 8 GitHub stars and 3 forks.

Related skills

Add Integration

8

Freddy-CHEN-ux/00_claude_skills

Build a new API integration for Nexus. Load when user mentions "add integration", "new integration", "integrate with", "connect to [service]", or "build [service] integration". Interactive workflow that discovers API endpoints, plans the integration, and creates a project for implementation.

Software Development

AI Agents

8

Freddy-CHEN-ux/00_claude_skills

Production-grade AI agent patterns with MCP integration, agentic RAG, handoff orchestration, multi-layer guardrails, and observability (modern best practices)

AI, LLMs & Agents

AI LLM

8

Freddy-CHEN-ux/00_claude_skills

Production LLM engineering skill. Covers strategy selection (prompting vs RAG vs fine-tuning), dataset design, PEFT/LoRA, evaluation workflows, deployment handoff to inference serving, and lifecycle operations with cost/safety controls.

AI, LLMs & Agents

AI Ml Data Science

8

Freddy-CHEN-ux/00_claude_skills

End-to-end data science patterns (modern best practices): problem framing -> data -> EDA -> feature engineering (with feature stores) -> modelling -> evaluation -> reporting, plus SQL transformation (SQLMesh). Emphasizes MLOps integration, drift monitoring, and production-ready workflows.

AI, LLMs & Agents

AI Ml Timeseries

8

Freddy-CHEN-ux/00_claude_skills

Operational patterns, templates, and decision rules for time series forecasting (modern best practices): tree-based methods (LightGBM), deep learning (Transformers, RNNs), future-guided learning, temporal validation, feature engineering, generative TS (Chronos), and production deployment. Emphasizes explainability, long-term dependency handling, and adaptive forecasting.

AI, LLMs & Agents

AI Mlops

8

Freddy-CHEN-ux/00_claude_skills

Complete MLOps skill covering production ML lifecycle and security. Includes data ingestion, model deployment, drift detection, monitoring, plus ML security (prompt injection, jailbreak defense, RAG security, privacy, governance). Modern automation-first patterns with multi-layered defenses.

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.