Ab Test Setup

Use when designing an A/B or split test: define the hypothesis, control and variants, estimate sample size, verify tracking, and predeclare metrics and stopping rules.

9 stars 4 forksTesting & QA

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add huytran1120/vietnam-agentic-skills --skill ab-test-setup
Manual install
git clone --depth 1 https://github.com/huytran1120/vietnam-agentic-skills.git /tmp/vietnam-agentic-skills
mkdir -p ~/.claude/skills
cp -r /tmp/vietnam-agentic-skills/skills/ab-test-setup ~/.claude/skills/ab-test-setup

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# A/B Test Setup

## 1️⃣ Purpose & Scope

Define an experiment that can answer a specific product question, and verify its assumptions before exposing users. This procedure cannot guarantee validity by itself.

- Documents the stopping rule
- Estimates sample needs under stated assumptions
- Makes the hypothesis and decision criteria reviewable

---

## 2️⃣ Pre-Requisites

You must have:

- A clear user problem
- Access to an analytics source
- Roughly estimated traffic volume

### Hypothesis Quality Checklist

A valid hypothesis includes:

- Observation or evidence
- Single, specific change
- Directional expectation
- Defined audience
- Measurable success criteria

---

## 3️⃣ Hypothesis Lock (Hard Gate)

Before designing variants or metrics, you MUST:

- Present the **final hypothesis**
- Specify:
  - Target audience
  - Primary metric
  - Expected direction of effect
  - Minimum Detectable Effect (MDE)

Use the hypothesis already agreed in the task. If a launch-critical choice is missing, present the concrete choice for confirmation while continuing independent analysis. Do not repeatedly request approval for a decision already authorized.

---

## 4️⃣ Assumptions & Validity Check (Mandatory)

Explicitly list assumptions about:

- Traffic stability
- User independence
- Metric reliability
- Randomization quality
- External factors (seasonality, campaigns, releases)

If assumptions are weak or violated:

- Warn the user
- Recommend delaying or redesigning the test

---
Read the full skill on GitHub

Details

Skill name
ab-test-setup
Path
skills/ab-test-setup/SKILL.md
License
MIT
Repo updated
29 October 2026
Copies
Found in 2 GitHub repos

FAQ

What does the Ab Test Setup skill do?

Use when designing an A/B or split test: define the hypothesis, control and variants, estimate sample size, verify tracking, and predeclare metrics and stopping rules.

How do I install the Ab Test Setup skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add huytran1120/vietnam-agentic-skills --skill ab-test-setup. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Ab Test Setup popular?

It lives in huytran1120/vietnam-agentic-skills, which has 9 GitHub stars and 4 forks.

Related skills

Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.