Agent Evals

Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates. Use when shipping agent features, validating prompt changes, or gating deployments on quality.

9 stars 4 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add huytran1120/vietnam-agentic-skills --skill agent-evals
Manual install
git clone --depth 1 https://github.com/huytran1120/vietnam-agentic-skills.git /tmp/vietnam-agentic-skills
mkdir -p ~/.claude/skills
cp -r /tmp/vietnam-agentic-skills/skills/agent-evals ~/.claude/skills/agent-evals

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Agent Evals

Create repeatable checks so agent behavior improves safely over time.

## When to Use This Skill

Use this skill when:
- Shipping new agent features or changing prompts
- Adding CI gates for agent quality and safety
- Building regression suites for tool-calling agents
- Measuring LLM output quality at scale
- Validating RAG retrieval accuracy

## Prerequisites

- Python 3.10+
- An LLM API key (OpenAI, Anthropic, etc.)
- pytest or a custom eval harness; `pytest-timeout` is required for the shown `--timeout` options
- Optional: Braintrust, Promptfoo, or LangSmith account

## Evaluation contract

The following are application templates, not bundled executable tests. Supply the `agent` adapter, fixtures, supported provider/model IDs, reviewed dependencies and test configuration. Approve provider cost and data transmission; keep fixtures synthetic and keys private. Sandbox tool effects and inspect tool/network traces, not only response words. Keyword refusals and numerical judge scores do not prove safety or factual accuracy. Define thresholds and human review before evaluating.

## Evaluation Layers

### Unit Evals — Prompt-Level Correctness

Test individual prompt → response quality:
Read the full skill on GitHub

Details

Skill name
agent-evals
Path
skills/agent-evals/SKILL.md
License
MIT
Repo updated
29 October 2026
Copies
Found in 2 GitHub repos

FAQ

What does the Agent Evals skill do?

Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates. Use when shipping agent features, validating prompt changes, or gating deployments on quality.

How do I install the Agent Evals skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add huytran1120/vietnam-agentic-skills --skill agent-evals. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Agent Evals popular?

It lives in huytran1120/vietnam-agentic-skills, which has 9 GitHub stars and 4 forks.

Related skills

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.