Flaky Test Killer

Find why a test is flaky and fix the root cause, never by adding retries, sleeps or bigger timeouts. Measures the flake rate with a confidence interval, runs the test under stressors (random order, timezone, fake clock at month end or DST, CPU load, fresh temp dir, low file limits), bisects for the test that pollutes it, and proves the fix with before and after numbers. Use when the user says "flaky test", "test passes locally fails in CI", "intermittent failure", "random test failure", "works on my machine", "test fails sometimes", "only fails in the full suite", or wants to quarantine or ret

517 stars 39 forksTesting & QA

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add dotslashsimran/simcraft --skill flaky-test-killer
Manual install
git clone --depth 1 https://github.com/dotslashsimran/simcraft.git /tmp/simcraft
mkdir -p ~/.claude/skills
cp -r /tmp/simcraft/skills/coding/flaky-test-killer ~/.claude/skills/flaky-test-killer

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Flaky test killer

A flaky test is a real bug: in the test, in the code, or in the environment. Retries hide it, and the same bug will later show up in production or in another test. The job is to make the failure happen on demand, name the cause, fix it, and prove with numbers that it is gone.

Scripts are Python 3.9+ standard library only. `S=~/simcraft/skills/coding/flaky-test-killer/scripts`.

## Rules

- **No retries, no sleeps, no longer timeouts, no `skip`** as the fix. A wait is fine only when it waits for the actual condition (an event, a join, a poll with a deadline).
- **Measure before you change anything.** Without a "before" rate you cannot prove an "after".
- **One cause at a time.** Many flaky tests have two bugs. Fix one, measure, and look again.
- **Keep an evidence log** (see below). Every claim needs a command and a number.

## Steps

1. **Reproduce and measure.** Run the single test, then the whole suite, many times.
   ```
   python3 $S/flake_rate.py -n 50 --json before.json -- <test command>
   python3 $S/flake_rate.py -n 200 -j 8 --json before.json -- <suite command with shuffle seed {seed}>
   ```
Read the full skill on GitHub

Details

Skill name
flaky-test-killer
Path
skills/coding/flaky-test-killer/SKILL.md
License
MIT
Repo updated
9 November 2026

FAQ

What does the Flaky Test Killer skill do?

Find why a test is flaky and fix the root cause, never by adding retries, sleeps or bigger timeouts. Measures the flake rate with a confidence interval, runs the test under stressors (random order, timezone, fake clock at month end or DST, CPU load, fresh temp dir, low file limits), bisects for the test that pollutes it, and proves the fix with before and after numbers. Use when the user says "flaky test", "test passes locally fails in CI", "intermittent failure", "random test failure", "works on my machine", "test fails sometimes", "only fails in the full suite", or wants to quarantine or ret

How do I install the Flaky Test Killer skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add dotslashsimran/simcraft --skill flaky-test-killer. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Flaky Test Killer popular?

It lives in dotslashsimran/simcraft, which has 517 GitHub stars and 39 forks.

Related skills

API Design Review

517

dotslashsimran/simcraft

Review the design of an API surface, meaning an HTTP API (OpenAPI spec or route code), an SDK's public classes and functions, or a CLI. Checks naming and consistency, resource modelling, error design (stable codes, RFC 9457 problem details, request ids), versioning and breaking changes, pagination, filtering, idempotency and retries, rate limits, timeouts, long-running operations, webhooks, auth, partial updates, ID, time and money formats, SDK ergonomics and CLI conventions. Uses scripts to lint OpenAPI files, classify changes between two spec versions as breaking or safe, and dump an SDK's p

Software Development39

Benchmark Honestly

517

dotslashsimran/simcraft

Build benchmarks that survive scrutiny and catch the classic lies (dead code elimination, lazy results, constant folding, warm caches, coordinated omission, averages hiding GC pauses, unequal work, noisy machines). Defines the question first, controls and discloses the environment, uses warmups, interleaved repeats, medians, bootstrap confidence intervals and Mann-Whitney tests, and ends with a one-line verdict that includes the CI. Bundles a command runner, an environment reporter and an open-loop latency tester. Use when the user says "benchmark this", "is this faster", "compare performance"

Software Development39

Bisect Anything

517

dotslashsimran/simcraft

Find the exact commit (or dependency version, config value, or input) that caused a regression by turning it into an automatic pass/fail check and running git bisect run in a separate worktree, then shrink the bad commit to the hunk that matters and explain it. Handles failing tests, changed output, slowdowns, crashes, visual changes, server behaviour, flaky checks, unbuildable commits and merge-heavy history. Use when the user says "what broke this", "find the commit that", "when did this start", "it used to work", "regression", "git bisect", "which version broke it", or "which change made it

Git & Code Review39

Cointegration Analysis

564

agiprolabs/claude-trading-skills

Cointegration testing for pairs trading using Engle-Granger, Johansen, and rolling stability analysis

Testing & QA113

Test Skill

291

joshuadavidthomas/opencode-agent-skills

A test skill to verify all plugin tools work correctly - use_skill, read_skill_file, run_skill_script, find_skills

Testing & QA32

Mean Reversion

564

agiprolabs/claude-trading-skills

Mean-reversion strategy tools including Hurst exponent, half-life estimation, z-score signals, ADF testing, and Ornstein-Uhlenbeck modeling

Testing & QA113

Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.