Make Evals

Create or maintain independent evaluations of model and AI agent failures in a new or existing repository. Use when asked to measure model behavior on incident-derived tasks with reproducible variants, deterministic checks, or rubric-based grading.

201 stars 12 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add swyxio/skills --skill make-evals
Manual install
git clone --depth 1 https://github.com/swyxio/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/make-evals ~/.claude/skills/make-evals

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Make evals

Evaluate a model or model-driven agent on tasks that expose observed failure modes. The subject is the model's response, decisions, or tool actions. Software tests can validate the harness, but do not count as model evals or evidence of model performance.

## Choose a mode and target

- **Setup:** create a suite in the requested repository, or a new neighboring repository if requested. Inspect existing conventions and reuse a suitable harness before adding one. If the target is unclear, infer it from the request; ask only when the choice would materially change the work.
- **Maintain/add:** inspect the existing cases, groups, harness, rubrics, baselines, and scoring before adding evaluations. Preserve unrelated work and organize new cases under the relevant failure modes.

In either mode, turn incident evidence into representative prompts, context, and, for agents, tool environments and tasks. Separate observed model failures from hypotheses. Define expected behavior and grading independently of the model's answer. Use small, sanitized fixtures and the simplest environment that preserves the relevant decision points. For example, test whether an agent diagnoses a stalled workflow and chooses an evidence-supported action; a coordinator unit test does not measure that behavior.

## Develop an incident-derived eval
Read the full skill on GitHub

Details

Skill name
make-evals
Repository
swyxio/skills
Path
make-evals/SKILL.md
License
MIT
Repo updated
16 November 2026

FAQ

What does the Make Evals skill do?

Create or maintain independent evaluations of model and AI agent failures in a new or existing repository. Use when asked to measure model behavior on incident-derived tasks with reproducible variants, deterministic checks, or rubric-based grading.

How do I install the Make Evals skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add swyxio/skills --skill make-evals. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Make Evals popular?

It lives in swyxio/skills, which has 201 GitHub stars and 12 forks.

Related skills

Accelevents API

201

swyxio/skills

Use when reading or updating AI Engineer Europe speaker records through the Accelevents REST API, especially for full-record PUT updates, auth header quirks, and preserving existing speaker fields.

Software Development12

Accelevents Speaker Sync

201

swyxio/skills

Use when website speaker, session, schedule, room, track, or headshot changes must be synchronized back to Accelevents for AI Engineer Europe 2026.

AI, LLMs & Agents12

Agents SDK

201

swyxio/skills

Build or debug stateful applications that specifically use Cloudflare's `agents` package, including Agent classes, routing, callable methods, synchronized state, scheduling, AIChatAgent, durable execution, or Agents SDK client hooks. Do not trigger for generic AI-agent design, the OpenAI Agents SDK, or ordinary Durable Object work that does not use the Cloudflare Agents SDK.

AI, LLMs & Agents12

Mastra Hono

191

jwynia/agent-skills

Develop AI agents, tools, and workflows with Mastra v1 Beta and Hono servers. This skill should be used when creating Mastra agents, defining tools with Zod schemas, building workflows with step data flow, setting up Hono API servers with Mastra adapters, or implementing agent networks. Keywords: mastra, hono, agent, tool, workflow, AI, LLM, typescript, API, MCP.

AI, LLMs & Agents24

LLM Optimized Content

150

guia-matthieu/clawfu-skills

Use when creating content that must be discoverable by AI search engines (ChatGPT, Perplexity, Gemini). Use when SEO alone isn't enough, when you need AI citations, or when optimizing for the "zero-click" future.

AI, LLMs & AgentsIn 1 list32

Mlflow 3

228

pproenca/dot-skills

MLflow 3 (open-source, pinned to 3.15) for classic-ML MLOps — logging and registering models, promoting versions across dev/staging/prod, standing up a tracking server, evaluating with gates, and serving. Corrects the MLflow 2-era defaults a model reaches for (artifact_path, registry stages and get_latest_versions, top-level mlflow.evaluate with baseline_model, runs-URI registration, pickle serialization, mlruns file stores, MLServer serving) with the MLflow 3 idioms that replaced them (named LoggedModels, aliases and copy_model_version, models.evaluate plus validate_evaluation_results, skops/

AI, LLMs & Agents19

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.