AI Prompt Regression Testing
Use when comparing prompt changes against a fixed evaluation set: record actual model settings, required checks, missing results, judge limitations and case-level regression evidence.
Install
npx skills add huytran1120/vietnam-agentic-skills --skill ai-prompt-regression-testinggit clone --depth 1 https://github.com/huytran1120/vietnam-agentic-skills.git /tmp/vietnam-agentic-skills
mkdir -p ~/.claude/skills
cp -r /tmp/vietnam-agentic-skills/skills/ai-prompt-regression-testing ~/.claude/skills/ai-prompt-regression-testingPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# AI Prompt Regression Testing ## When to Use - Compare a prompt change with an observed baseline on a fixed evaluation set. - Detect schema, factuality, refusal, tool-selection or task-completion regressions. - Record a reviewable decision when results are noisy, missing or contradictory. ## Workflow 1. Define the task, representative cases and forbidden outcomes. Use synthetic or approved/redacted data; freeze case IDs and the dataset revision before runs. 2. Record baseline/candidate prompt digests, exact available model/version, inference settings, tool versions and evaluation environment. Do not hardcode old model-family options or silently change models between comparisons. 3. Predeclare checks and acceptance rules. Use deterministic schema/field checks for structured output, behavioral checks for tools and a clear human/rubric standard for open text. Similarity does not prove factual or code correctness. 4. Run matched baseline/candidate cases under the same permissions and budget. Record latency/cost and repeated runs when variability matters. A timeout, missing output or judge error is incomplete, not a pass or a zero-cost result. 5. Treat judge outputs as evidence with limitations. Blind/randomize presentation where feasible, calibrate against human-labeled examples, and record judge identity/rubric. Do not ask a model to verify private reasoning or accept a judge's confidence as an objective correctness measurement.
Details
- Skill name
ai-prompt-regression-testing- Repository
- huytran1120/vietnam-agentic-skills
- Path
skills/ai-prompt-regression-testing/SKILL.md- License
- MIT
- Repo updated
- 29 October 2026
- Copies
- Found in 2 GitHub repos
FAQ
What does the AI Prompt Regression Testing skill do?
Use when comparing prompt changes against a fixed evaluation set: record actual model settings, required checks, missing results, judge limitations and case-level regression evidence.
How do I install the AI Prompt Regression Testing skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add huytran1120/vietnam-agentic-skills --skill ai-prompt-regression-testing. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is AI Prompt Regression Testing popular?
It lives in huytran1120/vietnam-agentic-skills, which has 9 GitHub stars and 4 forks.
Related skills
Apify Audience Analysis
9huytran1120/vietnam-agentic-skills
Understand audience demographics, preferences, behavior patterns, and engagement quality across Facebook, Instagram, YouTube, and TikTok.
Apify Competitor Intelligence
9huytran1120/vietnam-agentic-skills
Analyze competitor strategies, content, pricing, ads, and market positioning across Google Maps, Booking.com, Facebook, Instagram, YouTube, and TikTok.
Apify Content Analytics
9huytran1120/vietnam-agentic-skills
Track engagement metrics, measure campaign ROI, and analyze content performance across Instagram, Facebook, YouTube, and TikTok.
AI Red Teaming
9huytran1120/vietnam-agentic-skills
Run structured AI red team exercises for jailbreak resistance, data exfiltration risk, harmful output controls, and agent tool abuse resilience.
AI Product
9huytran1120/vietnam-agentic-skills
Design and evaluate LLM product features, retrieval, validated outputs, latency, and cost controls.
AI Security Hardening
9huytran1120/vietnam-agentic-skills
Harden AI/LLM deployments against prompt injection, data exfiltration, model theft, and supply chain attacks.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.