Flaky Test Killer
Find why a test is flaky and fix the root cause, never by adding retries, sleeps or bigger timeouts. Measures the flake rate with a confidence interval, runs the test under stressors (random order, timezone, fake clock at month end or DST, CPU load, fresh temp dir, low file limits), bisects for the test that pollutes it, and proves the fix with before and after numbers. Use when the user says "flaky test", "test passes locally fails in CI", "intermittent failure", "random test failure", "works on my machine", "test fails sometimes", "only fails in the full suite", or wants to quarantine or ret
Install
npx skills add dotslashsimran/simcraft --skill flaky-test-killergit clone --depth 1 https://github.com/dotslashsimran/simcraft.git /tmp/simcraft
mkdir -p ~/.claude/skills
cp -r /tmp/simcraft/skills/coding/flaky-test-killer ~/.claude/skills/flaky-test-killerPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Flaky test killer
A flaky test is a real bug: in the test, in the code, or in the environment. Retries hide it, and the same bug will later show up in production or in another test. The job is to make the failure happen on demand, name the cause, fix it, and prove with numbers that it is gone.
Scripts are Python 3.9+ standard library only. `S=~/simcraft/skills/coding/flaky-test-killer/scripts`.
## Rules
- **No retries, no sleeps, no longer timeouts, no `skip`** as the fix. A wait is fine only when it waits for the actual condition (an event, a join, a poll with a deadline).
- **Measure before you change anything.** Without a "before" rate you cannot prove an "after".
- **One cause at a time.** Many flaky tests have two bugs. Fix one, measure, and look again.
- **Keep an evidence log** (see below). Every claim needs a command and a number.
## Steps
1. **Reproduce and measure.** Run the single test, then the whole suite, many times.
```
python3 $S/flake_rate.py -n 50 --json before.json -- <test command>
python3 $S/flake_rate.py -n 200 -j 8 --json before.json -- <suite command with shuffle seed {seed}>
```Details
- Skill name
flaky-test-killer- Repository
- dotslashsimran/simcraft
- Path
skills/coding/flaky-test-killer/SKILL.md- License
- MIT
- Repo updated
- 9 November 2026
FAQ
What does the Flaky Test Killer skill do?
Find why a test is flaky and fix the root cause, never by adding retries, sleeps or bigger timeouts. Measures the flake rate with a confidence interval, runs the test under stressors (random order, timezone, fake clock at month end or DST, CPU load, fresh temp dir, low file limits), bisects for the test that pollutes it, and proves the fix with before and after numbers. Use when the user says "flaky test", "test passes locally fails in CI", "intermittent failure", "random test failure", "works on my machine", "test fails sometimes", "only fails in the full suite", or wants to quarantine or ret
How do I install the Flaky Test Killer skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add dotslashsimran/simcraft --skill flaky-test-killer. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Flaky Test Killer popular?
It lives in dotslashsimran/simcraft, which has 517 GitHub stars and 39 forks.
Related skills
API Design Review
517dotslashsimran/simcraft
Review the design of an API surface, meaning an HTTP API (OpenAPI spec or route code), an SDK's public classes and functions, or a CLI. Checks naming and consistency, resource modelling, error design (stable codes, RFC 9457 problem details, request ids), versioning and breaking changes, pagination, filtering, idempotency and retries, rate limits, timeouts, long-running operations, webhooks, auth, partial updates, ID, time and money formats, SDK ergonomics and CLI conventions. Uses scripts to lint OpenAPI files, classify changes between two spec versions as breaking or safe, and dump an SDK's p
Benchmark Honestly
517dotslashsimran/simcraft
Build benchmarks that survive scrutiny and catch the classic lies (dead code elimination, lazy results, constant folding, warm caches, coordinated omission, averages hiding GC pauses, unequal work, noisy machines). Defines the question first, controls and discloses the environment, uses warmups, interleaved repeats, medians, bootstrap confidence intervals and Mann-Whitney tests, and ends with a one-line verdict that includes the CI. Bundles a command runner, an environment reporter and an open-loop latency tester. Use when the user says "benchmark this", "is this faster", "compare performance"
Bisect Anything
517dotslashsimran/simcraft
Find the exact commit (or dependency version, config value, or input) that caused a regression by turning it into an automatic pass/fail check and running git bisect run in a separate worktree, then shrink the bad commit to the hunk that matters and explain it. Handles failing tests, changed output, slowdowns, crashes, visual changes, server behaviour, flaky checks, unbuildable commits and merge-heavy history. Use when the user says "what broke this", "find the commit that", "when did this start", "it used to work", "regression", "git bisect", "which version broke it", or "which change made it
Cointegration Analysis
564agiprolabs/claude-trading-skills
Cointegration testing for pairs trading using Engle-Granger, Johansen, and rolling stability analysis
Test Skill
291joshuadavidthomas/opencode-agent-skills
A test skill to verify all plugin tools work correctly - use_skill, read_skill_file, run_skill_script, find_skills
Mean Reversion
564agiprolabs/claude-trading-skills
Mean-reversion strategy tools including Hurst exponent, half-life estimation, z-score signals, ADF testing, and Ornstein-Uhlenbeck modeling
Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.