Skill Forge Benchmark

Benchmark Claude Code skill performance with variance analysis, tracking pass rate, execution time, and token usage across iterations. Runs multiple trials per eval for statistical reliability, aggregates results into benchmark.json, and generates comparison reports between skill versions. Use when user says "benchmark skill", "measure skill performance", "skill metrics", "compare skill versions", "skill performance", "track skill improvement", "skill regression test", or "skill A/B test".

189 stars 44 forksTesting & QA

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add AgriciDaniel/skill-forge --skill skill-forge-benchmark
Manual install
git clone --depth 1 https://github.com/AgriciDaniel/skill-forge.git /tmp/skill-forge
mkdir -p ~/.claude/skills
cp -r /tmp/skill-forge/skills/skill-forge-benchmark ~/.claude/skills/skill-forge-benchmark

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Skill Benchmarking & Performance Tracking

Measure and compare skill performance across iterations with statistical
rigor using multiple trials, variance analysis, and trend tracking.

## Process

### Step 1: Define Benchmark Configuration

Accept configuration as:
- **Existing eval set**: Path to `evals/evals.json` (from `/skill-forge eval`)
- **Benchmark config**: Custom config with trial count and thresholds

**Benchmark config schema:**
```json
{
  "skill_name": "my-skill",
  "skill_path": "./my-skill",
  "eval_set_path": "./evals/evals.json",
  "trials_per_eval": 3,
  "baseline_type": "no_skill",
  "previous_benchmark": null,
  "thresholds": {
    "min_pass_rate": 0.8,
    "max_avg_tokens": 100000,
    "max_avg_duration_seconds": 120,
    "min_improvement_ratio": 1.0
  }
}
```

### Step 2: Execute Benchmark Runs

For each eval, run `trials_per_eval` times (default: 3) to get reliable metrics:

1. Execute with-skill runs (3x per eval)
2. Execute baseline runs (3x per eval)
3. Capture per-run: pass/fail, token count, duration
4. Save each run's `timing.json` and `grading.json`

Use `agents/skill-forge-executor.md` for parallel execution where possible.

### Step 3: Aggregate Results

Run `python scripts/aggregate_benchmark.py <workspace>/iteration-<N> --skill-name <name>`:

**Output `benchmark.json` schema:**
Read the full skill on GitHub

Details

Skill name
skill-forge-benchmark
Path
skills/skill-forge-benchmark/SKILL.md
License
MIT
Repo updated
10 April 2026

FAQ

What does the Skill Forge Benchmark skill do?

Benchmark Claude Code skill performance with variance analysis, tracking pass rate, execution time, and token usage across iterations. Runs multiple trials per eval for statistical reliability, aggregates results into benchmark.json, and generates comparison reports between skill versions. Use when user says "benchmark skill", "measure skill performance", "skill metrics", "compare skill versions", "skill performance", "track skill improvement", "skill regression test", or "skill A/B test".

How do I install the Skill Forge Benchmark skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add AgriciDaniel/skill-forge --skill skill-forge-benchmark. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Skill Forge Benchmark popular?

It lives in AgriciDaniel/skill-forge, which has 189 GitHub stars and 44 forks.

Related skills

Skill Forge

189

AgriciDaniel/skill-forge

Ultimate Claude Code skill creator and architect. Designs, scaffolds, builds, reviews, evolves, and publishes production-grade Claude Code skills following the Agent Skills open standard and 3-layer architecture (directive, orchestration, execution). Handles single-file skills, multi-skill orchestrators with sub-skills and subagents, MCP-enhanced workflows, and full skill ecosystems. Industry detection for skill domain. Triggers on: "create skill", "build skill", "new skill", "skill creator", "skill builder", "skill-forge", "design skill", "scaffold skill", "review skill", "improve skill", "pu

Skill & Plugin Authoring44

Skill Forge Build

189

AgriciDaniel/skill-forge

Scaffold and build Claude Code skills from plans or descriptions. Generates SKILL.md files, sub-skills, scripts, references, agents, and templates following the Agent Skills standard. Use when user says "build skill", "scaffold skill", "generate skill", "create SKILL.md", or "implement skill".

AI, LLMs & Agents44

Skill Forge Convert

189

AgriciDaniel/skill-forge

Convert Claude Code skills to work on OpenAI Codex, Google Gemini CLI, Google Antigravity, and Cursor. Analyzes platform-specific features, generates target files (openai.yaml, AGENTS.md, GEMINI.md, .mdc rules), adapts frontmatter, converts MCP config, and produces compatibility reports. Use when user says "convert skill", "port skill", "multi-platform", "skill for codex", "skill for gemini", "skill for antigravity", "skill for cursor", "cross-platform skill", "convert to codex", "convert to gemini", "convert to antigravity", or "convert to cursor".

AI, LLMs & Agents44

Actions CI Tuning

423

mizchi/skills

Use when auditing or improving GitHub Actions workflows for a project. Covers cache setup (npm/pnpm/yarn), job parallelism, shard-based test splitting, artifact handling pitfalls, and Playwright-specific patterns. Trigger on: slow CI, cache miss, flaky shard jobs, merge-reports failures, or an explicit 'tune CI' request.

Testing & QA

Reality Testing Decisions

190

apple-ouyang/book-to-skill

Use when tempted to predict, analyze, or debate whether something will work instead of testing it. When facing uncertainty about a decision, product, hire, or opportunity and relying on gut feeling or expert opinions.

Testing & QA25

Ast Grep Practice

423

mizchi/skills

Operate ast-grep as a project lint tool. Covers sgconfig.yml, fix/rewrite rules, constraints, transform, testing, and CI. Use when writing rules ast-grep can express but general-purpose linters cannot.

Testing & QA

Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.