Agent Evaluation Engineering
Design reproducible evaluations for agent capabilities, trust, safety, trajectories, regressions, latency, and cost.
Install
npx skills add sandbaseai/workbuddy-skill --skill agent-evaluation-engineeringgit clone --depth 1 https://github.com/sandbaseai/workbuddy-skill.git /tmp/workbuddy-skill
mkdir -p ~/.claude/skills
cp -r /tmp/workbuddy-skill/skills/agent-evaluation-engineering ~/.claude/skills/agent-evaluation-engineeringPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# AI Agent Evaluation Engineering Use this skill when creating or reviewing an evaluation suite for an AI agent. Treat an evaluation as an executable product contract, not a collection of impressive demos. ## 1. Define the contract and risk Write down the user goal, allowed inputs, observable outputs, available tools, permissions, side effects, latency and cost limits, and acceptable escalation behavior. Rank failure modes by impact and likelihood. Keep capability quality separate from trust and safety: an agent can complete a task and still leak data, exceed authority, or make an irreversible change. ## 2. Build representative cases Create cases from real user journeys, production failures, domain edge cases, ambiguous requests, missing context, tool outages, permission boundaries, prompt injection, and unsafe requests. Include single-turn and multi-turn cases where memory, correction, or recovery matters. Partition cases into: - development examples for rapid iteration; - a held-out regression set that prompts and graders do not see during tuning; - adversarial cases focused on authority, privacy, injection, and irreversible effects. Turn every confirmed production failure into a minimized regression case. Remove duplicates and document why each case exists. ## 3. Specify evidence and graders For each case, define the expected invariant before running it. Prefer observable evidence: - final answer facts and citations;
Details
- Skill name
agent-evaluation-engineering- Repository
- sandbaseai/workbuddy-skill
- Path
skills/agent-evaluation-engineering/SKILL.md- License
- MIT
- Repo updated
- 8 September 2026
FAQ
What does the Agent Evaluation Engineering skill do?
Design reproducible evaluations for agent capabilities, trust, safety, trajectories, regressions, latency, and cost.
How do I install the Agent Evaluation Engineering skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add sandbaseai/workbuddy-skill --skill agent-evaluation-engineering. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Agent Evaluation Engineering popular?
It lives in sandbaseai/workbuddy-skill, which has 9 GitHub stars and 3 forks.
Related skills
Access Control Security Best Practices
9sandbaseai/workbuddy-skill
Design, review, implement, and verify least-privilege access controls across users, services, tenants, resources, and administrative workflows.
Accessibility Review
9sandbaseai/workbuddy-skill
Use when reviewing a web interface or interaction for semantic, keyboard, focus, screen-reader, ARIA, and contrast accessibility risks.
Acquire Codebase Knowledge
9sandbaseai/workbuddy-skill
Use when the user asks to map, document, or onboard into an existing codebase and needs evidence-backed architecture, structure, conventions, integrations, testing, and concerns documentation.
Agent Governance
9sandbaseai/workbuddy-skill
Use when governing an Agent that can call tools or delegate work, including policy-based allowlists, intent risk checks, approval gates, rate limits, trust scoring, content controls, and audit trails.
Agent Document Design
9sandbaseai/workbuddy-skill
Use when creating or editing Skills, agent instruction files, or referenced documents so triggers, steps, disclosure, completion criteria, and maintenance remain predictable.
Agent Observability
9sandbaseai/workbuddy-skill
Use when instrumenting or debugging an AI agent and you need privacy-aware traces, structured events, metrics, cost attribution, dashboards, alerts, or audit evidence.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.