Ab Test Stats

Calculate A/B test statistical significance. Use when: determining if test results are significant; calculating required sample size; estimating test duration; analyzing conversion experiments; making data-driven decisions

150 stars 32 forksTesting & QA

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats
Manual install
git clone --depth 1 https://github.com/guia-matthieu/clawfu-skills.git /tmp/clawfu-skills
mkdir -p ~/.claude/skills
cp -r /tmp/clawfu-skills/skills/analytics/ab-test-stats ~/.claude/skills/ab-test-stats

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# A/B Test Statistics Calculator

> Calculate statistical significance for A/B tests - know when your results are real, not random chance.

## When to Use This Skill

- **Test analysis** - Determine if results are statistically significant
- **Sample planning** - Calculate required sample size before testing
- **Duration estimation** - Know how long to run experiments
- **Power analysis** - Ensure tests can detect meaningful differences

## What Claude Does vs What You Decide

| Claude Does | You Decide |
|-------------|------------|
| Structures analysis frameworks | Metric definitions |
| Identifies patterns in data | Business interpretation |
| Creates visualization templates | Dashboard design |
| Suggests optimization areas | Action priorities |
| Calculates statistical measures | Decision thresholds |

## Dependencies

```bash
pip install scipy numpy click
```

## Commands

### Check Significance
```bash
python scripts/main.py significance --control 1000,50 --variant 1000,65
python scripts/main.py significance --control 5000,250 --variant 5000,300 --confidence 0.99
```

### Calculate Sample Size
```bash
python scripts/main.py sample-size --baseline 0.05 --mde 0.02
python scripts/main.py sample-size --baseline 0.10 --mde 0.01 --power 0.90
```

### Estimate Duration
```bash
python scripts/main.py duration --traffic 1000 --baseline 0.05 --mde 0.02
```

## Examples

### Example 1: Analyze Test Results
Read the full skill on GitHub

Details

Skill name
ab-test-stats
Path
skills/analytics/ab-test-stats/SKILL.md
License
MIT
Repo updated
1 October 2026

FAQ

What does the Ab Test Stats skill do?

Calculate A/B test statistical significance. Use when: determining if test results are significant; calculating required sample size; estimating test duration; analyzing conversion experiments; making data-driven decisions

How do I install the Ab Test Stats skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add guia-matthieu/clawfu-skills --skill ab-test-stats. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Ab Test Stats popular?

It lives in guia-matthieu/clawfu-skills, which has 150 GitHub stars and 32 forks, and is included in 1 curated skill list.

Related skills

Aarrr Metrics

150

guia-matthieu/clawfu-skills

Measure and optimize growth using the AARRR (Pirate Metrics) framework with stage-specific KPIs and funnel analysis

Marketing, Sales & BusinessIn 1 list32

Account Health

150

guia-matthieu/clawfu-skills

Assess customer account health using product usage, support sentiment, payment status, and relationship signals

Finance & CryptoIn 1 list32

Ad Spend Optimizer

150

guia-matthieu/clawfu-skills

Analyze paid advertising performance across channels and recommend budget reallocation to maximize ROAS and minimize CAC. Use when: planning quarterly ad budget allocation, diagnosing underperforming ad channels, deciding whether to scale spend on a channel, calculating marginal ROI across Google Ads, Meta, LinkedIn, or TikTok, rebalancing media mix after performance shifts, or setting up a test-and-scale framework for new channels.

Marketing, Sales & BusinessIn 1 list32

Acceptance Pipeline Catalog

228

pproenca/dot-skills

Use when implementing, reviewing, or debugging a Gherkin acceptance-test pipeline with mutation testing. Covers parser, JSON IR, generator, runtime, step handlers, test runner, mutator, value mutation rules, execution, result classification, reporting, project layout, conformance, and agent setup. Based on Uncle Bob's Acceptance Pipeline Specification. Trigger even when the user mentions Gherkin parsing, acceptance test generation, mutation testing for acceptance tests, or building a portable test pipeline.

Testing & QA19

Stakeholder QA Prep

110

microsoft/cat-agent-skills

Anticipates the questions senior stakeholders will ask in an upcoming review and pre-drafts suggested answers, each anchored to a specific retrieved number. Generates questions across five categories — number-driven, forward-looking, risk-driven, comparative, and operational — weighted toward the named audience, and marks unknowns rather than guessing. Use when the user asks "what will the CFO ask", "prep me for questions in the review", "anticipate board questions on these numbers", "build a Q&A prep sheet", or "what tough questions should I expect". Do NOT use for building the review pre-rea

Testing & QA122

Dx Harness

228

pproenca/dot-skills

Developer-experience friction auditing and fixing — slow onboarding, repeated manual setup steps, missing bootstrap/reset/seed scripts, undiscoverable conventions. Audits the repo, scores findings, scaffolds canonical fixes (bootstrap.sh, reset.sh, seed.sh, AGENTS.md, task-runner entries), then verifies the harness end-to-end in a scratch worktree against a 60-second time-to-first-commit target. Triggers on phrases like "audit dx", "fix dev friction", "time to first commit", "set up the harness", "I keep doing X manually", "every time I reset the db I have to...", and on new-repo bootstrapping

Testing & QA19

Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.