Agentic Evaluation Framework

Builds LLM and agent evals: weighted rubrics, LLM-as-judge design, inter-rater agreement and pairwise ranking (Elo, Bradley-Terry) from scores you collect. Use when setting up a judge, scoring outputs, or comparing model or prompt variants.

1,157 stars 217 forksAI, LLMs & Agents Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add borghei/Claude-Skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add borghei/Claude-Skills --skill agentic-evaluation-framework
Manual install
git clone --depth 1 https://github.com/borghei/Claude-Skills.git /tmp/Claude-Skills
mkdir -p ~/.claude/skills
cp -r /tmp/Claude-Skills/engineering/agentic-evaluation-framework ~/.claude/skills/agentic-evaluation-framework

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Agentic Evaluation Framework

> **Category:** Engineering
> **Domain:** AI Engineering

## Overview

Design and run trustworthy evaluations for LLM and agent outputs: pick the right grading method (programmatic check, LLM-as-judge, or human review), write a scoring rubric that judges can apply consistently, rank competing variants by pairwise comparison, and watch for the biases that quietly corrupt judge scores — position bias, verbosity bias, and self-preference. The goal is an eval that you can *trust enough to ship on*: calibrated against human labels, cheap enough to run on every change, and tracked alongside cost and latency so you never trade quality away by accident. This skill is model- and vendor-agnostic: it reasons about the evaluation *method*, not any one provider's API, and its scripts aggregate scores you have already collected — they never call a model.

## Clarify First

Before designing or running an evaluation, settle the inputs below. When one is missing or vague and would change the result, ask about it; when it wouldn't, use a sensible default and say which one you used.

- **What "good" means:** the dimensions you care about (accuracy, helpfulness, safety, format, tool-use) and their relative weight (defines the rubric `criteria` and `weights`).
Read the full skill on GitHub

Details

Skill name
agentic-evaluation-framework
Path
engineering/agentic-evaluation-framework/SKILL.md
Repo updated
8 November 2026

FAQ

What does the Agentic Evaluation Framework skill do?

Builds LLM and agent evals: weighted rubrics, LLM-as-judge design, inter-rater agreement and pairwise ranking (Elo, Bradley-Terry) from scores you collect. Use when setting up a judge, scoring outputs, or comparing model or prompt variants.

How do I install the Agentic Evaluation Framework skill?

Claude Code plugin: /plugin marketplace add borghei/Claude-Skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Agentic Evaluation Framework popular?

It lives in borghei/Claude-Skills, which has 1,157 GitHub stars and 217 forks, and is included in 1 curated skill list.

Related skills

Cto Advisor

1.2k

borghei/Claude-Skills

Technical leadership guidance for engineering teams, architecture decisions, and technology strategy. Use when assessing technical debt, scaling engineering teams, evaluating technologies, or making architecture decisions.

Software DevelopmentIn 1 list217

Customer Success Manager

1.2k

borghei/Claude-Skills

Monitors customer health, predicts churn risk, and identifies expansion opportunities using weighted scoring models for SaaS customer success

Marketing, Sales & BusinessIn 1 list217

Financial Analyst

1.2k

borghei/Claude-Skills

Performs financial ratio analysis, DCF valuation, budget variance analysis, and rolling forecast construction for strategic decision-making

Finance & CryptoIn 1 list217

Agents in the Team

1.2k

borghei/Claude-Skills

Run delivery when AI coding and ops agents take tickets. Use when setting agent delegation policy, writing agent-ready tickets, planning review capacity, measuring agent vs human delivery, or rolling agents out safely.

AI, LLMs & AgentsIn 1 list217

Agenthub

1.2k

borghei/Claude-Skills

Runs a multi-agent job as a dependency graph: validates the DAG, tracks each agent's state on a board, then ranks and merges their outputs. Use when spawning parallel agents, monitoring a run, or merging results.

AI, LLMs & AgentsIn 1 list217

AI Act Readiness

1.2k

borghei/Claude-Skills

EU AI Act readiness assessment and sprint playbook. Use when preparing for the high-risk AI system deadlines (Dec 2027 / Aug 2028), when a notified-body conformity assessment is scheduled, or when GPAI (general-purpose AI) obligations apply.

AI, LLMs & AgentsIn 1 list217

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.