Task Based Eval

Evaluate whether fresh agents achieve a stated goal using a tool, document, interface, skill or workflow. Organize definitions by task under evals/, specify inputs, permitted actions, outputs and success criteria, then improve the supplied resource from independent outcome checks and rubric grading.

423 stars 6 forksProductivity & Notes

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add mizchi/skills --skill task-based-eval
Manual install
git clone --depth 1 https://github.com/mizchi/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/task-based-eval ~/.claude/skills/task-based-eval

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Task-based evaluation

Measure whether an agent can achieve a goal using something supplied to it.
The target may be a tool, document, application, interface, skill or workflow.
Observe actual inputs, actions, feedback, outputs and resulting state, then grade
goal achievement. API usability is one application of this method.

```text
goal + input + supplied resource + permitted actions
  → fresh executor → observed interaction + output/state
  → goal verification + rubric → targeted change → fresh trial
```

Improve the supplied resource or environment while keeping the task distribution,
executor model, budgets and other conditions fixed during a comparison. This is
task-based evaluation; measuring a resource's usefulness through downstream task
outcomes is also called extrinsic evaluation. Read
[research and terminology](references/research.md) for the distinction from
behavioral testing, API usability and reflective optimization.

For a grading-only or planning-only request, perform that phase using the supplied
evidence and budget. Label the narrower result; do not silently start a full experiment.

## Define a task, then cases and trials

Use `evals/<task-name>/` for a stable goal, such as `route-support-request` or
`repair-invalid-config`. The directory names the goal, not the tool, model,
revision or execution. A **case** is a particular input/initial state for that
goal. A **trial** is one fresh attempt on a case. See
Read the full skill on GitHub

Details

Skill name
task-based-eval
Repository
mizchi/skills
Path
task-based-eval/SKILL.md
Repo updated
18 November 2026

FAQ

What does the Task Based Eval skill do?

Evaluate whether fresh agents achieve a stated goal using a tool, document, interface, skill or workflow. Organize definitions by task under evals/, specify inputs, permitted actions, outputs and success criteria, then improve the supplied resource from independent outcome checks and rubric grading.

How do I install the Task Based Eval skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add mizchi/skills --skill task-based-eval. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Task Based Eval popular?

It lives in mizchi/skills, which has 423 GitHub stars and 6 forks.

Related skills

Actions CI Tuning

423

mizchi/skills

Use when auditing or improving GitHub Actions workflows for a project. Covers cache setup (npm/pnpm/yarn), job parallelism, shard-based test splitting, artifact handling pitfalls, and Playwright-specific patterns. Trigger on: slow CI, cache miss, flaky shard jobs, merge-reports failures, or an explicit 'tune CI' request.

Testing & QA

AI Index

423

mizchi/skills

Method and tooling for measuring how AI-generated a piece of prose reads, in Japanese or English. Two layers — a deterministic lint for removable surface tells (scripts/slopscore.py, no API) and a judgment layer of typed probabilistic questions (Jev noul/score) for the tells that survive editing. Includes measured human/AI baselines, a gap-first calibration procedure, and labeled fixtures. Use when building or calibrating an AI-likeness score, or when a draft needs a defensible number rather than a vibe. For writing guidance use natural-writing-ja / natural-writing-en instead.

AI, LLMs & Agents

Apm Usage

423

mizchi/skills

Reference for APM manifests, installation, targeted updates, deployment paths and lockfiles. Invoke when the user explicitly asks about APM or apm.yml, or an unfamiliar APM command/error needs exact CLI details. Do not automatically invoke for every project that happens to have installed skills.

DevOps & Cloud

21 Day Self Interview

108

Forlives/21-day-self-interview

夜间自我访谈引导(21 天)。扮演资深存在主义心理咨询师,每晚提三个有意义的问题,记录回答并在节点回映,帮助用户看清自己。Use when running a nightly self-inquiry / journaling ritual driven by a scheduled task.

Productivity & Notes35

Project Planner

99

adrianpuiu/claude-skills-marketplace

Comprehensive project planning and documentation generator for software projects. Creates structured requirements documents, system design documents, and task breakdown plans with implementation tracking. Use when starting a new project, defining specifications, creating technical designs, or breaking down complex systems into implementable tasks. Supports user story format, acceptance criteria, component design, API specifications, and hierarchical task decomposition with requirement traceability.

Productivity & Notes14

Slm Bot Memory

278

qualixar/superlocalmemory

Cross-bot memory on a shared computer (Grok Bot, Cursor plugin, any host where several agents share one machine). Explains agent_id attribution vs. profile/scope access, how to namespace by agent_id + profile + scope, and what must never be written to memory. Read this before the first remember/recall on a new Grok-Bot-style host.

Productivity & NotesIn 1 list36

Browse the most popular Claude skills, all productivity & notes skills, or write your own with the skills generator.