LLM Finetuning

Use when the user asks to fine-tune an open model with LoRA or QLoRA, PEFT, SFT, instruction tuning, TRL SFTTrainer, DPO, train on my data or merge an adapter. Produces a go/no-go decision, chat-format dataset spec, training config, eval plan and serving steps. Not for multi-node training (use distributed-training) or post-training quantization (use llm-model-quantization).

45 stars 6 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add j4flmao/agent-skills --skill llm-finetuning
Manual install
git clone --depth 1 https://github.com/j4flmao/agent-skills.git /tmp/agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/agent-skills/skills/ai/llm-finetuning ~/.claude/skills/llm-finetuning

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# LLM Fine-Tuning (PEFT / LoRA)

## Purpose
Fine-tuning changes model behaviour (format, style, task skill, domain language) from examples. It is the wrong tool for injecting frequently changing facts; use retrieval for that. Parameter-efficient methods such as LoRA train small low-rank adapters on frozen base weights, making single-GPU training of 7-8B models routine and letting one base model serve many adapters.

## Agent Protocol

### Trigger
"fine-tune", "LoRA", "QLoRA", "PEFT", "SFT", "instruction tuning", "DPO", "preference tuning", "adapter", "merge_and_unload".

### Input Context
- Goal and why prompting/RAG is insufficient (with failing examples).
- Dataset: size, source, license, format, PII status.
- Base model and license; GPU type/VRAM; serving target.
- Eval set and success metric.

### Output Artifact
Go/no-go decision, dataset spec, training config, eval plan, export/serving instructions.

### Response Format
```
Decision: {fine-tune|RAG|prompting} because {reason}
Base model: {id} (license {x})  Method: {LoRA|QLoRA|full}  Stage: {SFT|DPO}
Data: {n} examples, chat format, split {train/val/test}
LoRA: r={r} alpha={a} dropout={p} targets={modules}
Train: lr={lr} epochs={e} eff_batch={b} max_len={L}
Eval: base vs tuned on {metric}; regression suite {name}
Export: {adapter|merged} -> {engine}
```

### Completion Criteria
- [ ] Baseline (prompted base model, RAG) measured first.
- [ ] Held-out test set never used for tuning decisions.
Read the full skill on GitHub

Details

Skill name
llm-finetuning
Path
skills/ai/llm-finetuning/SKILL.md
License
MIT
Repo updated
17 November 2026

FAQ

What does the LLM Finetuning skill do?

Use when the user asks to fine-tune an open model with LoRA or QLoRA, PEFT, SFT, instruction tuning, TRL SFTTrainer, DPO, train on my data or merge an adapter. Produces a go/no-go decision, chat-format dataset spec, training config, eval plan and serving steps. Not for multi-node training (use distributed-training) or post-training quantization (use llm-model-quantization).

How do I install the LLM Finetuning skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add j4flmao/agent-skills --skill llm-finetuning. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is LLM Finetuning popular?

It lives in j4flmao/agent-skills, which has 45 GitHub stars and 6 forks.

Related skills

Agent Observability

45

j4flmao/agent-skills

Use when the user asks about agent observability, tracing agent reasoning, debugging an agent loop, OpenTelemetry GenAI spans for LLM calls, LLM cost tracking per agent, agent audit logs or hallucination/eval monitoring in production. Produces instrumentation config, trace schema, dashboards, alerts and cost attribution. Not for generic service tracing without LLMs (use devops-opentelemetry).

DevOps & Cloud

AI Agent Architectures

45

j4flmao/agent-skills

Use when the user asks to build an AI agent, design an agent that calls tools and keeps memory, write an agent system prompt or pick an agent framework (Claude Agent SDK, OpenAI Agents SDK, LangGraph). Produces an architecture brief covering loop, tools, memory, guardrails and evals, routing depth to child skills. Not for workflow patterns (use ai-agentic-workflows).

AI, LLMs & Agents

AI Agent Core Loop

45

j4flmao/agent-skills

Use when the user asks about the agent loop, ReAct vs plan-and-solve, re-planning, Reflexion, an agent stuck in a loop, max iterations or termination conditions. Produces a loop spec or patch with state, success predicate, caps and no-progress detection. Not for overall agent design (use ai-agent-architectures) or tool schemas (use tool-calling-principles).

AI, LLMs & Agents

LLM Kv Cache Paged Attention

45

j4flmao/agent-skills

Use when the user asks about KV cache size, PagedAttention, vLLM OOM, gpu_memory_utilization, max_num_seqs, prefix caching, GQA/MLA memory or long-context OOM. Produces KV-per-token math, max concurrency and vLLM flags. Not for weight quantization (use llm-model-quantization) or local llama.cpp setups (use local-slm-inference).

AI, LLMs & Agents

Genai Vision

45

j4flmao/agent-skills

Use when the user asks to generate images with Stable Diffusion, SDXL, FLUX or a diffusers pipeline, img2img, inpainting, ControlNet, LoRA styles, or transcribe audio with Whisper. Produces model choice with license note, pipeline code, memory settings and serving outline. Not for VLM image understanding (use multimodal-pipelines) or detection models (use ml-computer-vision).

AI, LLMs & Agents

LLM Model Quantization

45

j4flmao/agent-skills

Use when the user asks to quantize a model, GGUF, Q4_K_M, llama-quantize, GPTQ, AWQ, NF4, bitsandbytes, INT8/FP8 weights or a model too big for the GPU. Produces format choice per engine, quantization commands, memory estimate and an eval gate. Not for NVFP4/FP8 kernel-level tuning (use extreme-quantization-mastery) or QLoRA training (use llm-finetuning).

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.