Ab Test Designer

Design A/B tests with hypotheses, variants, sample sizes, and analysis plans. TRIGGERS - Use when user wants to run A/B tests, split tests, or experiment with variations.

39 stars 5 forksTesting & QA Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add lionelsimai/claude-skills-collection

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add lionelsimai/claude-skills-collection --skill ab-test-designer
Manual install
git clone --depth 1 https://github.com/lionelsimai/claude-skills-collection.git /tmp/claude-skills-collection
mkdir -p ~/.claude/skills
cp -r /tmp/claude-skills-collection/skills/ab-test-designer ~/.claude/skills/ab-test-designer

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# A/B Test Designer

## Overview
Designs rigorous A/B tests with clear hypotheses, variant specifications, sample size calculations, and analysis plans.

## Workflow

### Step 1: Define the Test
1. **What are you testing?** (page, email, ad, feature)
2. **Current metric**: What's the baseline performance?
3. **Goal**: What improvement are you hoping for?
4. **Traffic/volume**: How many users/emails/impressions per day?

### Step 2: Structure the Test

## Output Format

```markdown
# A/B Test: [Test Name]

## Hypothesis
If we [change X], then [metric Y] will [increase/decrease] by [Z%] because [reasoning].

## Test Details
- **Type**: A/B / A/B/C / Multivariate
- **Primary metric**: [what you're measuring]
- **Secondary metrics**: [supporting metrics]
- **Guardrail metrics**: [what shouldn't get worse]

## Variants

### Control (A)
[Description of current experience]

### Variant (B)
[Description of the change]
[Mockup/wireframe description if applicable]

## Sample Size & Duration
- **Baseline conversion**: [X%]
- **Minimum detectable effect**: [X%]
- **Statistical significance**: 95%
- **Required sample size**: [N per variant]
- **Estimated duration**: [X days]

## Analysis Plan
1. Wait for minimum sample size before checking
2. Check primary metric first
3. Segment analysis: [segments to check]
4. If significant: [implementation plan]
5. If not significant: [next steps]

## Risks & Considerations
- [Risk 1]: [mitigation]
- [Risk 2]: [mitigation]
```

## Quality Checklist
Read the full skill on GitHub

Details

Skill name
ab-test-designer
Path
skills/ab-test-designer/SKILL.md
License
MIT
Repo updated
8 November 2026
Copies
Found in 2 GitHub repos

FAQ

What does the Ab Test Designer skill do?

Design A/B tests with hypotheses, variants, sample sizes, and analysis plans. TRIGGERS - Use when user wants to run A/B tests, split tests, or experiment with variations.

How do I install the Ab Test Designer skill?

Claude Code plugin: /plugin marketplace add lionelsimai/claude-skills-collection. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Ab Test Designer popular?

It lives in lionelsimai/claude-skills-collection, which has 39 GitHub stars and 5 forks.

Related skills

Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.