Prompt Version Management

Manage prompt versions, run A/B tests across agent prompts, track performance regressions, and safely roll out prompt changes in production. Covers prompt diffing, semantic versioning, canary releases, and automated evaluation.

492 stars 68 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add cosmicstack-labs/mercury-agent-skills --skill prompt-version-management
Manual install
git clone --depth 1 https://github.com/cosmicstack-labs/mercury-agent-skills.git /tmp/mercury-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/mercury-agent-skills/categories/ai-ml/prompt-version-management ~/.claude/skills/prompt-version-management

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Prompt Version Management & A/B Testing

## Overview

Prompts are code — and they need version control, testing, and staged rollouts just like software. A single changed word can swing accuracy by 20%. This skill covers how to manage prompt versions systematically, run controlled experiments, and deploy prompt changes with confidence.

---

## Core Concepts

### Why Prompt Versioning Matters

| Problem | Without Versioning | With Versioning |
|---------|-------------------|-----------------|
| A prompt change breaks behavior | No way to roll back | Instant rollback to previous SHA |
| "Which prompt is in production?" | Check Slack history | Single source of truth |
| A/B test needed | Manual, error-prone | Structured experiment framework |
| Regression from edit | Undetected until users complain | Automated eval suite catches it |
| Collaboration | Merge conflicts in shared docs | PR-based workflow with reviews |

### Prompt Version Schema

```
prompts/
├── agents/
│   ├── support-agent/
│   │   ├── system-prompt-v1.0.0.md
│   │   ├── system-prompt-v1.1.0.md
│   │   ├── system-prompt-v2.0.0-beta.md
│   │   └── system-prompt-v2.0.0.md
│   └── research-agent/
│       └── ...
├── shared/
│   ├── guardrails-v1.0.0.md
│   └── output-format-v2.0.0.md
└── experiments/
    ├── exp-2024-01-fewshot-vs-cot/
    │   ├── control.md
    │   └── variant.md
    └── ...
```

### Semantic Versioning for Prompts

| Bump | When | Example |
|------|------|---------|
Read the full skill on GitHub

Details

Skill name
prompt-version-management
Path
categories/ai-ml/prompt-version-management/SKILL.md
License
MIT
Repo updated
25 August 2026

FAQ

What does the Prompt Version Management skill do?

Manage prompt versions, run A/B tests across agent prompts, track performance regressions, and safely roll out prompt changes in production. Covers prompt diffing, semantic versioning, canary releases, and automated evaluation.

How do I install the Prompt Version Management skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add cosmicstack-labs/mercury-agent-skills --skill prompt-version-management. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Prompt Version Management popular?

It lives in cosmicstack-labs/mercury-agent-skills, which has 492 GitHub stars and 68 forks.

Related skills

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.