Waxa Eval

Use when iterating on a skill's prompt with the waxa CLI (https://github.com/mizchi/skills/tree/main/tools/waxa) — authoring scenarios, choosing graders, interpreting unclear-points, advancing a ledger, and judging convergence. Encodes the four-stage iteration pattern observed in real iter loops (structural fix → grader breadth → surface-form coverage → residual unclear) and the scenario-design pitfalls (blank-slate executor's network limit, prompt expectation explicitness, regex coverage of Japanese/English surface forms). Meta-skill: do NOT auto-invoke for routine skill edits; fires only whe

423 stars 6 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add mizchi/skills --skill waxa-eval
Manual install
git clone --depth 1 https://github.com/mizchi/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/waxa-eval ~/.claude/skills/waxa-eval

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# waxa-eval

Empirical evaluation loop for skill prompts, codified from real iter runs.
This skill is the operating manual for `waxa`; the CLI itself lives in
`tools/waxa/` (see its README for argument-level reference).

`waxa` extends [microsoft/waza](https://github.com/microsoft/waza) with
`empirical-prompt-tuning` semantics on top: forced Self-report, ledger
across iterations, four grader types, and convergence detection.

## When to invoke

Explicit user request only:

- "evaluate this skill with waxa"
- "iterate on `<skill>` until it converges"
- "add a waxa scenario for `<scenario>`"
- "interpret these unclear-points / Self-report"

When NOT to use:

- One-off skill edits (typo fix, tightening one sentence) — direct edit, no eval cycle.
- A skill is brand-new and has no scenarios yet — author the SKILL.md first; only run eval once you have a stable target to measure.
- The user did not ask for evaluation — meta-skill, no auto-invoke.

## The four-stage iteration pattern

Repeated across the `skill-selector`, `nix-setup`, and `skill-finder` evals. Treat it as the default trajectory; it converges in 3-4 iterations when the skill is structurally sound.

| Stage | What you fix | Typical Iter | Symptom of being on this stage |
|---|---|---|---|
| 1. Structural fix | A whole behavior is missing — the skill never tells the executor to do X. | 1 → 2 | LLM grader and surface graders **both** fail; pass rate jumps after one edit. |
Read the full skill on GitHub

Details

Skill name
waxa-eval
Repository
mizchi/skills
Path
waxa-eval/SKILL.md
Repo updated
18 November 2026

FAQ

What does the Waxa Eval skill do?

Use when iterating on a skill's prompt with the waxa CLI (https://github.com/mizchi/skills/tree/main/tools/waxa) — authoring scenarios, choosing graders, interpreting unclear-points, advancing a ledger, and judging convergence. Encodes the four-stage iteration pattern observed in real iter loops (structural fix → grader breadth → surface-form coverage → residual unclear) and the scenario-design pitfalls (blank-slate executor's network limit, prompt expectation explicitness, regex coverage of Japanese/English surface forms). Meta-skill: do NOT auto-invoke for routine skill edits; fires only whe

How do I install the Waxa Eval skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add mizchi/skills --skill waxa-eval. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Waxa Eval popular?

It lives in mizchi/skills, which has 423 GitHub stars and 6 forks.

Related skills

Actions CI Tuning

423

mizchi/skills

Use when auditing or improving GitHub Actions workflows for a project. Covers cache setup (npm/pnpm/yarn), job parallelism, shard-based test splitting, artifact handling pitfalls, and Playwright-specific patterns. Trigger on: slow CI, cache miss, flaky shard jobs, merge-reports failures, or an explicit 'tune CI' request.

Testing & QA

AI Index

423

mizchi/skills

Method and tooling for measuring how AI-generated a piece of prose reads, in Japanese or English. Two layers — a deterministic lint for removable surface tells (scripts/slopscore.py, no API) and a judgment layer of typed probabilistic questions (Jev noul/score) for the tells that survive editing. Includes measured human/AI baselines, a gap-first calibration procedure, and labeled fixtures. Use when building or calibrating an AI-likeness score, or when a draft needs a defensible number rather than a vibe. For writing guidance use natural-writing-ja / natural-writing-en instead.

AI, LLMs & Agents

Apm Usage

423

mizchi/skills

Reference for APM manifests, installation, targeted updates, deployment paths and lockfiles. Invoke when the user explicitly asks about APM or apm.yml, or an unfamiliar APM command/error needs exact CLI details. Do not automatically invoke for every project that happens to have installed skills.

DevOps & Cloud

Benchmark Model

213

modular/skills

Benchmark a model served on MAX with the `max benchmark` command: measure throughput (tokens/sec), latency (TTFT, TPOT, inter-token latency), and GPU utilization by driving load against a running `max serve` endpoint. Use this whenever the user wants to benchmark, load-test, or measure the performance of a MAX model, get tokens-per-second / TTFT / TPOT numbers, run a concurrency or request-rate sweep, compare latency vs throughput, size a deployment, or produce benchmark JSON, even if they don't say "benchmark" by name. Also use when a `max benchmark` run fails to connect or reports zero/garba

AI, LLMs & Agents22

Skill Finder

423

mizchi/skills

Use ONLY when the user explicitly asks to discover or evaluate a skill from OUTSIDE the curated catalog — e.g., "find a Stripe skill", "is there a skill for X?", "evaluate this candidate before adopting". Meta-skill, complementary to skill-selector: do NOT auto-invoke on routine apm-management, project-init, or catalog-resident requests. Cross-source survey across vetted registries (Anthropic official, claude-skill-registry, VoltAgent/awesome-agent-skills, ComposioHQ, Superpowers, GitHub topic) with a mandatory waxa-eval adoption gate.

AI, LLMs & Agents

Debug Model

213

modular/skills

Debug a MAX model that loads, compiles, serves, and generates tokens but whose output disagrees with a reference implementation. Use whenever parity debugging stalls on scalar taps, the model returns gibberish or wrong greedy tokens, logit cosine is high but argmax differs, or generation is coherent then diverges. Applies during an architecture port, a quantization bring-up, a multi-GPU conversion, or after a MAX upgrade. Triggers on "parity failure", "silent corruption", "logits match but tokens diverge", "top-1 mismatch", "greedy divergence", and "model serves but generates garbage". Not for

AI, LLMs & Agents22

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.