Evaluate Skill

Run, read, and diagnose a skill's Caliper eval — its success rate over k attempts, whether it fires, and whether it beats the agent without it. Use when the user wants to run, validate, interpret, or compare a skill's eval, or write an .eval.yaml spec whose tasks they have already decided.

227 stars 25 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)

npx skills add edonadei/caliper --skill evaluate-skill

Manual install

git clone --depth 1 https://github.com/edonadei/caliper.git /tmp/caliper
mkdir -p ~/.claude/skills
cp -r /tmp/caliper/skills/evaluate-skill ~/.claude/skills/evaluate-skill

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Evaluate Skill

Operate Caliper: run a skill's eval, read what the results say, and trace each failure to the place that fixes it.

## Prerequisites

The `caliper` CLI must be on `PATH`. This skill can be copied into an agent without the Caliper repo, so install it if missing:

```bash
pipx install caliper-eval
```

`caliper <command> --help` is the authority on flags. [REFERENCE.md](REFERENCE.md) covers the full spec format, engines, and what the results mean.

## The four questions

A skill eval answers four separate questions. Each has its own measurement and its own fix location, so keep them apart: a blended number hides which half broke.

| Question | Measured by | A failure is fixed in |
|---|---|---|
| **Fires**: does the agent reach for the skill when it should, and only then? | `activates:` on tasks, and trigger probes | the skill's `description` frontmatter |
| **Works**: once it fires, does it get the job done? | `expect:` / `assert:`, scored as the success rate | the skill's body |
| **Earns**: does it beat the agent without it? | the control (`--ablate <skill-name>`: the declared neighbourhood minus this skill) and `caliper compare`. For a truly bare agent, ablate every declared skill and every declared `mcp:` server, and isolate the run | the tasks: if the control passes too, the task is too easy |
| **Holds**: does it stay good across edits and over time? | `caliper compare` of each full run against the previous one, skill drift | the edit that moved it |

Read the full skill on GitHub

Details

Skill name: evaluate-skill
Repository: edonadei/caliper
Path: skills/evaluate-skill/SKILL.md
License: MIT
Repo updated: 27 October 2026
Listed in: VoltAgent/awesome-agent-skills

FAQ

What does the Evaluate Skill skill do?

How do I install the Evaluate Skill skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add edonadei/caliper --skill evaluate-skill. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Evaluate Skill popular?

It lives in edonadei/caliper, which has 227 GitHub stars and 25 forks, and is included in 1 curated skill list.

Related skills

Oil Cover

285

oil-oil/oil-cover

生成小红书和 B 站 AI 工具实操视频封面，支持脚本模式与 Agent 自主模式、真实视频证据、三画幅输出和可选创作者头像。仅在用户明确提到 oil-cover、$oil-cover、使用 oil-cover 或指定本 Skill 时触发；普通封面、首图、视频封面请求不要自动触发。

AI, LLMs & Agents29

Agenttrace Session Audit

152

luoyuctl/agenttrace

Audit local AI coding-agent sessions with agenttrace. Use when the user asks to inspect Claude Code, Codex CLI, Qwen Code, Cline, Aider, Cursor exports, Hermes Agent, OpenCode, OpenClaw, Pi, Oh My Pi, Kimi CLI, Copilot-style logs, or generic JSON/JSONL traces for cost, tokens, tool failures, latency, anomalies, health, diffs, or CI gates.

AI, LLMs & AgentsIn 3 lists

Project Cairn

252

iBlinkQ/project-cairn

Standardize how an AI-collaboration project turns work into reusable knowledge. Use when initializing or retrofitting Project Cairn in a project, recording progress after meaningful work, maintaining AGENTS/CLAUDE/cairn docs, auditing project knowledge for drift or missing records, pulling and citing external knowledge, or graduating validated project experience into a reusable knowledge base.

AI, LLMs & Agents48

AI Media Generator

266

Hao0321/ai-media-generator

為使用者產生高品質的 AI 生圖、生影片、生音樂提示詞，並在需要時透過瀏覽器自動化實際送到目標平台。涵蓋 OiiOii、Kling 3.0/O-series、Seedance 2.0/2.5、Suno v5.5、Seedream 5.0/4.0、Vidu Q3、Midjourney V8.1、Flux 1.1 Pro / Kontext、Runway Gen-4.5 / Aleph、Google Veo 3.1、Ideogram 3、Nano Banana Pro、Stable Diffusion 3.5（⚠️ OpenAI Sora 2 已於 2026-04-26 停運，API 撐到 2026-09-24，預設改推 Runway/Veo/Kling）。只要使用者提到「AI 生圖」「AI 影片」「AI 音樂」「做 MV」「做 storyboard」「寫 prompt 給 XXX」「我想用 Kling/Suno/Midjourney/Runway/Veo...」「幫我操作 OiiOii / 即夢 / 可靈」「txt2img / img2video / 文生圖 / 文生影片 / 圖生影片」「角色一致性」「多鏡頭分鏡」「運鏡」「結果有瑕疵 / 不夠精緻 / 怎麼修」，或任何跟上述平台或影像/影片/音樂生成工作流相關的任務，都要用這個 skill。即使他們沒講明平台，只要任務是要餵給某個生

AI, LLMs & Agents62

AI Anime Series

233

10291124jcjc-dotcom/ai-anime-series

AI 动画单元剧：定好一组原创动画角色（比如胖仓鼠和长腊肠犬），以后每集说一句主题，就写出 15 秒 / 30 秒 / 1 分钟的无台词动作喜剧剧本，自动拼出角色设定图、宫格故事板和即梦 Seedance 视频提示词（参考图编号、宫格数、每格秒数自动对齐），用户在即梦画布里出图出视频，再检查成片、改单个情节、记录已拍内容、拼合集。用户说「做一个像呱呱历险记那样的 AI 动画号」「AI 动画短剧」「动画单元剧」「出一集 XX 和 XX」「故事板直出」「九宫格分镜做动画」，或者要改已有一集的剧情、镜头、提示词时使用。Make a recurring-character AI animated comedy series, one self-contained episode at a time, with Jimeng (Dreamina) Seedance omni-reference.

AI, LLMs & Agents54

Google AI Mode Skill

293

PleasePrompto/google-ai-mode-skill

Use this skill when the user requests current information, documentation, coding examples, or web research beyond the knowledge cutoff. Queries Google's AI Search mode to retrieve comprehensive AI-generated overviews with source citations from 100+ websites. Returns markdown with inline footnoted references [1][2][3]. You will receive a detailed Markdown file with references and information summarized directly from Google's AI search. Ideal for you to get new information and clues for further research.

AI, LLMs & Agents43

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.