Extreme Quantization Mastery

Use when the user asks about FP8, INT4, NVFP4, MXFP4, GPTQ, AWQ, SmoothQuant, W4A16, W8A8, NF4 or bitsandbytes for LLMs on NVIDIA GPUs. Produces a quantization recipe, code and accuracy-eval plan matched to the GPU and serving engine. Not for format/GGUF basics (use llm-model-quantization) or TensorRT engines (use tensorrt-optimization).

45 stars 6 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add j4flmao/agent-skills --skill extreme-quantization-mastery
Manual install
git clone --depth 1 https://github.com/j4flmao/agent-skills.git /tmp/agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/agent-skills/skills/hardware-acceleration/extreme-quantization ~/.claude/skills/extreme-quantization-mastery

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Extreme Quantization (sub-16-bit LLMs)

## Purpose
Shrink LLM weights, activations and KV cache to 8, 4 or fewer bits so models fit in less memory and run faster. Weight-only quantization speeds up memory-bound decoding; weight+activation quantization speeds up compute-bound prefill and large batches. Every scheme must be validated on task evals, not just perplexity.

## Agent Protocol

### Trigger
"quantize", "int4", "int8", "fp8", "fp4", "nvfp4", "mxfp4", "gptq", "awq", "smoothquant", "w4a16", "w8a8", "nf4", "bitsandbytes", "kv cache fp8".

### Input Context
- Model + size, target GPU generation (Ampere sm_80, Ada sm_89, Hopper sm_90, Blackwell sm_100/sm_120).
- Serving engine (vLLM, SGLang, TensorRT-LLM, llama.cpp) and batch profile (latency vs throughput).
- Accuracy budget and evaluation set.

### Output Artifact
Quantization recipe + command/code + evaluation plan. No file unless requested.

### Response Format
```
Scheme: {W4A16|W8A8-INT8|W8A8-FP8|NVFP4|...} group={size}
Method: {GPTQ|AWQ|SmoothQuant|RTN|dynamic FP8}
Hardware path: {kernel/tensor-core type}
Memory: {before} -> {after}
Eval: {tasks, max allowed drop}
```

### Completion Criteria
- [ ] Target engine + GPU support the chosen format natively (otherwise it is slower, not faster).
- [ ] Calibration data resembles production prompts.
- [ ] Baseline vs quantized evaluated on the same harness and seeds.
Read the full skill on GitHub

Details

Skill name
extreme-quantization-mastery
Path
skills/hardware-acceleration/extreme-quantization/SKILL.md
License
MIT
Repo updated
17 November 2026

FAQ

What does the Extreme Quantization Mastery skill do?

Use when the user asks about FP8, INT4, NVFP4, MXFP4, GPTQ, AWQ, SmoothQuant, W4A16, W8A8, NF4 or bitsandbytes for LLMs on NVIDIA GPUs. Produces a quantization recipe, code and accuracy-eval plan matched to the GPU and serving engine. Not for format/GGUF basics (use llm-model-quantization) or TensorRT engines (use tensorrt-optimization).

How do I install the Extreme Quantization Mastery skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add j4flmao/agent-skills --skill extreme-quantization-mastery. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Extreme Quantization Mastery popular?

It lives in j4flmao/agent-skills, which has 45 GitHub stars and 6 forks.

Related skills

Agent Observability

45

j4flmao/agent-skills

Use when the user asks about agent observability, tracing agent reasoning, debugging an agent loop, OpenTelemetry GenAI spans for LLM calls, LLM cost tracking per agent, agent audit logs or hallucination/eval monitoring in production. Produces instrumentation config, trace schema, dashboards, alerts and cost attribution. Not for generic service tracing without LLMs (use devops-opentelemetry).

DevOps & Cloud

AI Agent Architectures

45

j4flmao/agent-skills

Use when the user asks to build an AI agent, design an agent that calls tools and keeps memory, write an agent system prompt or pick an agent framework (Claude Agent SDK, OpenAI Agents SDK, LangGraph). Produces an architecture brief covering loop, tools, memory, guardrails and evals, routing depth to child skills. Not for workflow patterns (use ai-agentic-workflows).

AI, LLMs & Agents

AI Agent Core Loop

45

j4flmao/agent-skills

Use when the user asks about the agent loop, ReAct vs plan-and-solve, re-planning, Reflexion, an agent stuck in a loop, max iterations or termination conditions. Produces a loop spec or patch with state, success predicate, caps and no-progress detection. Not for overall agent design (use ai-agent-architectures) or tool schemas (use tool-calling-principles).

AI, LLMs & Agents

Genai Vision

45

j4flmao/agent-skills

Use when the user asks to generate images with Stable Diffusion, SDXL, FLUX or a diffusers pipeline, img2img, inpainting, ControlNet, LoRA styles, or transcribe audio with Whisper. Produces model choice with license note, pipeline code, memory settings and serving outline. Not for VLM image understanding (use multimodal-pipelines) or detection models (use ml-computer-vision).

AI, LLMs & Agents

Edge AI

45

j4flmao/agent-skills

Use when the user asks about edge AI, on-device inference, TensorRT on Jetson, ONNX Runtime, LiteRT (TFLite), TinyML, int8 quantization, Coral, NPUs or ExecuTorch. Produces export, quantization and on-device benchmarking plans. Not for model training (use ml-deep-learning) or cloud LLM serving (use ml-inference-optimization).

AI, LLMs & Agents

LLM Finetuning

45

j4flmao/agent-skills

Use when the user asks to fine-tune an open model with LoRA or QLoRA, PEFT, SFT, instruction tuning, TRL SFTTrainer, DPO, train on my data or merge an adapter. Produces a go/no-go decision, chat-format dataset spec, training config, eval plan and serving steps. Not for multi-node training (use distributed-training) or post-training quantization (use llm-model-quantization).

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.