Model Quantization

Expert skill for AI model quantization and optimization. Covers 4-bit/8-bit quantization, GGUF conversion, memory optimization, and quality-performance tradeoffs for deploying LLMs in resource-constrained JARVIS environments.

48 stars 4 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add martinholovsky/claude-skills-generator --skill model-quantization
Manual install
git clone --depth 1 https://github.com/martinholovsky/claude-skills-generator.git /tmp/claude-skills-generator
mkdir -p ~/.claude/skills
cp -r /tmp/claude-skills-generator/skills/model-quantization ~/.claude/skills/model-quantization

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Model Quantization Skill

> **File Organization**: Split structure. See `references/` for detailed implementations.

## 1. Overview

**Risk Level**: MEDIUM - Model manipulation, potential quality degradation, resource management

You are an expert in AI model quantization with deep expertise in 4-bit/8-bit optimization, GGUF format conversion, and quality-performance tradeoffs. Your mastery spans quantization techniques, memory optimization, and benchmarking for resource-constrained deployments.

You excel at:
- 4-bit and 8-bit model quantization (Q4_K_M, Q5_K_M, Q8_0)
- GGUF format conversion for llama.cpp
- Quality vs. performance tradeoff analysis
- Memory footprint optimization
- Quantization impact benchmarking

**Primary Use Cases**:
- Deploying LLMs on consumer hardware for JARVIS
- Optimizing models for CPU/GPU memory constraints
- Balancing quality and latency for voice assistant
- Creating model variants for different hardware tiers

---

## 2. Core Principles

1. **TDD First** - Write tests before quantization code; verify quality metrics pass
2. **Performance Aware** - Optimize for memory, latency, and throughput from the start
3. **Quality Preservation** - Minimize perplexity degradation for use case
4. **Security Verified** - Always validate model checksums before loading
5. **Hardware Matched** - Select quantization based on deployment constraints

---

## 3. Core Responsibilities

### 3.1 Quality-Preserving Optimization

When quantizing models, you will:
Read the full skill on GitHub

Details

Skill name
model-quantization
Path
skills/model-quantization/SKILL.md
License
Unlicense
Repo updated
6 December 2025

FAQ

What does the Model Quantization skill do?

Expert skill for AI model quantization and optimization. Covers 4-bit/8-bit quantization, GGUF conversion, memory optimization, and quality-performance tradeoffs for deploying LLMs in resource-constrained JARVIS environments.

How do I install the Model Quantization skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add martinholovsky/claude-skills-generator --skill model-quantization. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Model Quantization popular?

It lives in martinholovsky/claude-skills-generator, which has 48 GitHub stars and 4 forks.

Related skills

API Expert

48

martinholovsky/claude-skills-generator

Expert API architect specializing in RESTful API design, GraphQL, gRPC, and API security. Deep expertise in OpenAPI 3.1, authentication patterns (OAuth2, JWT), rate limiting, pagination, and OWASP API Security Top 10. Use when designing scalable APIs, implementing API gateways, or securing API endpoints.

Software Development

Applescript

48

martinholovsky/claude-skills-generator

Expert in AppleScript and JavaScript for Automation (JXA) for macOS system scripting. Specializes in secure script execution, application automation, and system integration. HIGH-RISK skill due to shell command execution and system-wide control capabilities.

Productivity & Notes

Appsec Expert

48

martinholovsky/claude-skills-generator

Elite Application Security engineer specializing in secure SDLC, OWASP Top 10 2025, SAST/DAST/SCA integration, threat modeling (STRIDE), and vulnerability remediation. Expert in security testing, cryptography, authentication patterns, and DevSecOps automation. Use when securing applications, implementing security controls, or conducting security assessments.

Security

Model Split Screen

38

vargHQ/skills

Make the "same prompt, two models" comparison video that every model launch produces (Claude Fable 5.2 vs Opus 5.5, Seedance vs Kling, Eevee vs Cycles): run one prompt twice under the same rules, log the real time and cost of each run, then lay the two results side by side (1920×1080) or stacked (1080×1920), started on the same frame, each with its name and its time and cost under it, optionally blind ("A" and "B" until the last seconds, then the reveal) and with a winner frame. scripts/split.py does the layout with ffmpeg and Pillow, so it works with an ffmpeg built without drawtext. Use when

AI, LLMs & AgentsIn 1 list

LLM Integration

48

martinholovsky/claude-skills-generator

Expert skill for integrating local Large Language Models using llama.cpp and Ollama. Covers secure model loading, inference optimization, prompt handling, and protection against LLM-specific vulnerabilities including prompt injection, model theft, and denial of service attacks.

AI, LLMs & Agents

Prompt Engineering

48

martinholovsky/claude-skills-generator

Expert skill for prompt engineering and task routing/orchestration. Covers secure prompt construction, injection prevention, multi-step task orchestration, and LLM output validation for JARVIS AI assistant.

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.