Ab Test Setup

Design and run statistically rigorous A/B tests and experiments. Use when planning experiments, calculating sample sizes, designing test variants, selecting metrics, analyzing results, or when someone says "let's test that.

1,157 stars 217 forksTesting & QA Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add borghei/Claude-Skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add borghei/Claude-Skills --skill ab-test-setup
Manual install
git clone --depth 1 https://github.com/borghei/Claude-Skills.git /tmp/Claude-Skills
mkdir -p ~/.claude/skills
cp -r /tmp/Claude-Skills/product-team/ab-test-setup ~/.claude/skills/ab-test-setup

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# A/B Test Setup - Experimentation Design & Analysis

**Category:** Product Team
**Tags:** A/B testing, experiments, statistical significance, sample size, feature flags, hypothesis testing

## Overview

A/B Test Setup provides the complete framework for designing experiments that produce statistically valid, actionable results. Most A/B tests fail not because the variant was wrong, but because the test was poorly designed: wrong sample size, wrong metric, or someone peeked at results and stopped early. This skill prevents those mistakes.

---

## Clarify First

Before designing the experiment, settle the inputs below. When one is missing or vague and would change the result, ask about it; when it wouldn't, use a sensible default and say which one you used.

- **Primary metric + minimum detectable effect:** the one success metric and smallest lift worth detecting (drives sample size, duration, and metric selection).
- **Baseline conversion rate:** current rate for the primary metric (sets required sample size per variant).
- **Available traffic to the test surface:** daily eligible visitors (decides whether the test is feasible or needs a bolder change / qualitative method).
- **The change and its rationale:** what varies and the data behind it (drives the hypothesis and variant design).

Ask about the two or three inputs that matter most for this request, not the whole list. If the user wants a result straight away, produce it and state your assumptions at the top.
Read the full skill on GitHub

Details

Skill name
ab-test-setup
Path
product-team/ab-test-setup/SKILL.md
Repo updated
8 November 2026

FAQ

What does the Ab Test Setup skill do?

Design and run statistically rigorous A/B tests and experiments. Use when planning experiments, calculating sample sizes, designing test variants, selecting metrics, analyzing results, or when someone says "let's test that.

How do I install the Ab Test Setup skill?

Claude Code plugin: /plugin marketplace add borghei/Claude-Skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Ab Test Setup popular?

It lives in borghei/Claude-Skills, which has 1,157 GitHub stars and 217 forks, and is included in 1 curated skill list.

Related skills

Cto Advisor

1.2k

borghei/Claude-Skills

Technical leadership guidance for engineering teams, architecture decisions, and technology strategy. Use when assessing technical debt, scaling engineering teams, evaluating technologies, or making architecture decisions.

Software DevelopmentIn 1 list217

Customer Success Manager

1.2k

borghei/Claude-Skills

Monitors customer health, predicts churn risk, and identifies expansion opportunities using weighted scoring models for SaaS customer success

Marketing, Sales & BusinessIn 1 list217

Financial Analyst

1.2k

borghei/Claude-Skills

Performs financial ratio analysis, DCF valuation, budget variance analysis, and rolling forecast construction for strategic decision-making

Finance & CryptoIn 1 list217

API Test Suite Builder

1.2k

borghei/Claude-Skills

Generates API test skeletons from an OpenAPI spec, measures which endpoints tests really assert on, and checks response samples against the spec. Use when adding API tests, auditing API test coverage, or catching contract drift.

Testing & QAIn 1 list217

Ab Test Setup

1.2k

borghei/Claude-Skills

Design and analyze A/B tests: sample size, test duration, and statistical significance for conversion experiments. Use when setting up an A/B test, calculating sample size, designing an experiment, or analyzing results.

Testing & QAIn 1 list217

Brainstorm Experiments

1.2k

borghei/Claude-Skills

Designs cheap experiments with a pre-set pass bar (at least X% of Y will do Z) and picks the method. Use when validating an idea before building, testing willingness to pay, or choosing between fake door, landing page and A/B test.

Testing & QAIn 1 list217

Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.