Prompt Regression Suite
Design a regression test suite that catches an LLM feature getting worse when the prompt, model, or context changes. Use when asked to stop prompt changes breaking production, set up golden tests or CI gates for an LLM feature, or test a model/prompt upgrade before shipping it. Produces a golden case set, per-case pass criteria, CI gate thresholds, and a triage protocol for failures. For designing first-time evaluation of a new feature use ai-eval-plan instead.
Install
/plugin marketplace add mohitagw15856/pm-claude-skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add mohitagw15856/pm-claude-skills --skill prompt-regression-suitegit clone --depth 1 https://github.com/mohitagw15856/pm-claude-skills.git /tmp/pm-claude-skills
mkdir -p ~/.claude/skills
cp -r /tmp/pm-claude-skills/plugins/pm-agentops/skills/prompt-regression-suite ~/.claude/skills/prompt-regression-suitePersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Prompt Regression Suite Skill Every prompt tweak, model upgrade, and context change is a deploy. This skill designs the suite that runs on each one and answers a single question: *did anything that used to work stop working?* ## What This Skill Produces - A **golden case set**: curated inputs with per-case pass criteria - **Scoring methods** per case class (exact, rubric-judge, property checks) - **CI gate thresholds** — what blocks a merge vs. what warns - A **failure triage protocol** — flaky vs. regressed vs. golden-set-wrong ## Required Inputs Ask for (if not already provided): - **The feature and its contract** — what the LLM step receives and must produce - **What has broken before** (or nearly) — past incidents seed the best cases - **Real traffic examples** — 10-20 representative inputs, including ugly ones - **What triggers a run** — prompt edits, model bumps, retrieval changes, all of the above? ## Building the Golden Set Compose the set from four deliberate classes — not a random sample: | Class | Purpose | Share | |---|---|---| | **Core paths** | The 5-10 inputs that represent most real traffic | ~40% | | **Past failures** | Every input that caused a bug, complaint, or incident — permanently | ~25% | | **Edge & adversarial** | Empty/huge inputs, wrong language, injection attempts, off-topic | ~25% | | **Canaries** | Cases pinned to behaviours you never want to change (refusals, format, tone) | ~10% |
Details
- Skill name
prompt-regression-suite- Repository
- mohitagw15856/pm-claude-skills
- Path
plugins/pm-agentops/skills/prompt-regression-suite/SKILL.md- License
- MIT
- Repo updated
- 18 November 2026
- Listed in
- BehiSecc/awesome-claude-skills
FAQ
What does the Prompt Regression Suite skill do?
Design a regression test suite that catches an LLM feature getting worse when the prompt, model, or context changes. Use when asked to stop prompt changes breaking production, set up golden tests or CI gates for an LLM feature, or test a model/prompt upgrade before shipping it. Produces a golden case set, per-case pass criteria, CI gate thresholds, and a triage protocol for failures. For designing first-time evaluation of a new feature use ai-eval-plan instead.
How do I install the Prompt Regression Suite skill?
Claude Code plugin: /plugin marketplace add mohitagw15856/pm-claude-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Prompt Regression Suite popular?
It lives in mohitagw15856/pm-claude-skills, which has 1,573 GitHub stars and 281 forks, and is included in 1 curated skill list.
Related skills
Ab Test Planner
1.6kmohitagw15856/pm-claude-skills
Diseña pruebas A/B estadísticamente rigurosas para características de producto, cambios de UI, flujos de onboarding y experimentos de precios. Úsalo cuando te pidan configurar un experimento, diseñar una prueba A/B, calcular tamaño de muestra o interpretar resultados de pruebas. Produce un plan de prueba completo con hipótesis, definición de variantes, tamaño de muestra, estimación de duración, métricas de seguridad y una guía de interpretación de resultados.
Ab Test Planner
1.6kmohitagw15856/pm-claude-skills
प्रोडक्ट फ़ीचर, UI बदलाव, ऑनबोर्डिंग फ़्लो और प्राइसिंग प्रयोगों के लिए सांख्यिकीय रूप से कठोर A/B टेस्ट डिज़ाइन करें। जब कोई प्रयोग सेट करने, A/B टेस्ट डिज़ाइन करने, सैंपल साइज़ निकालने या टेस्ट के नतीजे समझाने को कहे, तब इसका उपयोग करें। परिकल्पना, वेरिएंट की परिभाषा, सैंपल साइज़, अवधि का अनुमान, गार्डरेल मेट्रिक और नतीजों की व्याख्या की गाइड के साथ एक पूरा टेस्ट प्लान तैयार करता है।
Accessible Travel Planner
1.6kmohitagw15856/pm-claude-skills
Summarise what Accessible Travel Planner does in one line. Use when asked to [trigger phrases the user would say]. Produces [the concrete artifact].
RAG Architecture Review
1.6kmohitagw15856/pm-claude-skills
Review an existing Retrieval-Augmented Generation system and find why it underperforms. Use when asked to review or audit a RAG pipeline, diagnose wrong/ungrounded answers from a 'chat with your docs' feature, or improve an already-built knowledge assistant. Produces a staged review — ingestion, chunking, retrieval, reranking, generation, evaluation — with prioritised findings, root causes, and concrete fixes.
Prompt Optimizer
1.6kmohitagw15856/pm-claude-skills
Diagnose and rewrite an underperforming LLM prompt so it produces reliable, well-structured output. Use when asked to improve a prompt, fix a prompt that gives inconsistent or wrong results, reduce hallucination/refusals, or make output follow a format. Produces a rewritten prompt with a diagnosis of what was failing, the specific changes and why, and a small test set to verify the fix.
RAG Design Doc
1.6kmohitagw15856/pm-claude-skills
Design a Retrieval-Augmented Generation system end to end. Use when asked to design a RAG pipeline, a 'chat with your docs' feature, a knowledge assistant, or to debug why a RAG system gives wrong/ungrounded answers. Produces a RAG design doc — ingestion & chunking, embeddings & index, retrieval & reranking, the generation prompt, grounding/citations, evaluation, and failure modes with mitigations.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.