Evals Router

Use when evaluating LLM or RAG outputs: audit eval coverage, analyze failed traces, write binary judge prompts, validate judges against labels, generate targeted synthetic cases, evaluate retrieval quality, or plan review tooling. Do not use for ordinary software test implementation.

9 stars 4 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add jscraik/Agent-Skills --skill evals-router
Manual install
git clone --depth 1 https://github.com/jscraik/Agent-Skills.git /tmp/Agent-Skills
mkdir -p ~/.claude/skills
cp -r /tmp/Agent-Skills/Skills/agent-ops/evals-router ~/.claude/skills/evals-router

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Evals Router

Route LLM/RAG eval work through the smallest dependency-ordered proof plan.
Start with the proof closest to the changed claim and keep package, scenario,
scorer, runtime, observability, hosted, and external authorities separate.

## When To Use

Use for eval coverage, failed traces, judge prompts, labels, RAG evidence,
synthetic cases, evaluator profiles, scenario contracts, or review tooling. Do
not use for ordinary software test implementation.

## Inputs

Need the user goal, claim being evaluated, candidate identity, and available
traces, labels, scorecard, prompt, retrieval evidence, scenarios, receipts, or
target artifacts.

Discover repository-owned evidence before asking: inspect current source,
schemas, reports, receipts, runtime state, and user-named artifacts. Ask one
question only when this inspection cannot resolve a choice that would
materially change the route.

For release-impacting or multi-lane work, also inventory the candidate revision
or digest, scenario set, rubric version, runner, execution profile,
provider/model or deterministic identity, generated state, and authority of
each receipt.

## Outputs

Primary artifacts include a patched eval, judge prompt, trace analysis,
synthetic case, RAG check, review-interface spec, eval-contract repair, or
blocked report. Add only the receipts needed to support the user's claim.

Artifact shapes:

- eval-audit: `claim_id -> case_id|gap_id`
Read the full skill on GitHub

Details

Skill name
evals-router
Path
Skills/agent-ops/evals-router/SKILL.md
License
Apache-2.0
Repo updated
16 November 2026

FAQ

What does the Evals Router skill do?

Use when evaluating LLM or RAG outputs: audit eval coverage, analyze failed traces, write binary judge prompts, validate judges against labels, generate targeted synthetic cases, evaluate retrieval quality, or plan review tooling. Do not use for ordinary software test implementation.

How do I install the Evals Router skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add jscraik/Agent-Skills --skill evals-router. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Evals Router popular?

It lives in jscraik/Agent-Skills, which has 9 GitHub stars and 4 forks.

Related skills

1password

9

jscraik/Agent-Skills

Plan, diagnose, and validate 1Password CLI workflows. Use when tasks need op CLI sign-in, secret references, op run, op inject, item reads, env injection, or service-account secret access.

Security

Agentation

9

jscraik/Agent-Skills

Audit, validate, and troubleshoot Agentation integrations in frontend apps. Use when annotations, MCP registration, endpoint sync, webhook delivery, or watch mode readiness are failing.

AI, LLMs & Agents

Agents Md

9

jscraik/Agent-Skills

Use when reviewing, creating, shrinking, or refactoring AGENTS.md and directly linked instruction guidance for stale or conflicting rules against current model documentation, scoped routing, deduplication, or progressive disclosure.

AI, LLMs & Agents

Fix Mise

9

jscraik/Agent-Skills

Diagnose, fix, and validate mise runtime failures. Use when commands fail from mise config, missing runtimes, stale pins, trust prompts, or shell activation drift.

AI, LLMs & Agents

Elixir Pro

9

jscraik/Agent-Skills

Create and review idiomatic Elixir code with OTP patterns, supervision trees, and Phoenix LiveView. Use when building or debugging Elixir services that need reliable concurrency and fault tolerance.

AI, LLMs & Agents

Goal Governor

9

jscraik/Agent-Skills

Use when a Codex goal/task is stuck, hanging, not finishing, or needs status. Reads goal.md, state.yaml, receipts.jsonl; syncs reported status with board files; fixes invalid state.yaml; classifies blockers; decides done. Not for ordinary reviews or one-off fixes.

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.