Advanced Evaluation

Design LLM-as-judge rubrics, direct scores and pairwise comparisons. Use for judge models, evals, position bias and calibration against human labels.

9 stars 4 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add huytran1120/vietnam-agentic-skills --skill advanced-evaluation
Manual install
git clone --depth 1 https://github.com/huytran1120/vietnam-agentic-skills.git /tmp/vietnam-agentic-skills
mkdir -p ~/.claude/skills
cp -r /tmp/vietnam-agentic-skills/skills/advanced-evaluation ~/.claude/skills/advanced-evaluation

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Advanced Evaluation

This skill covers production-grade techniques for evaluating LLM outputs using LLMs as judges. It synthesizes research from academic papers, industry practices, and practical implementation experience into actionable patterns for building reliable evaluation systems.

**Key insight**: LLM-as-a-Judge is not a single technique but a family of approaches, each suited to different evaluation contexts. Choosing the right approach and mitigating known biases is the core competency this skill develops.

## When to Use
Activate this skill when:

- Building automated evaluation pipelines for LLM outputs
- Comparing multiple model responses to select the best one
- Establishing consistent quality standards across evaluation teams
- Debugging evaluation systems that show inconsistent results
- Designing A/B tests for prompt or model changes
- Creating rubrics for human or automated evaluation
- Analyzing correlation between automated and human judgments

## Core Concepts

### The Evaluation Taxonomy

Evaluation approaches fall into two primary categories with distinct reliability profiles:

**Direct Scoring**: A single LLM rates one response on a defined scale.
- Best for: Objective criteria (factual accuracy, instruction following, toxicity)
- Reliability: Moderate to high for well-defined criteria
- Failure mode: Score calibration drift, inconsistent scale interpretation

**Pairwise Comparison**: An LLM compares two responses and selects the better one.
Read the full skill on GitHub

Details

Skill name
advanced-evaluation
Path
skills/advanced-evaluation/SKILL.md
License
MIT
Repo updated
29 October 2026
Copies
Found in 2 GitHub repos

FAQ

What does the Advanced Evaluation skill do?

Design LLM-as-judge rubrics, direct scores and pairwise comparisons. Use for judge models, evals, position bias and calibration against human labels.

How do I install the Advanced Evaluation skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add huytran1120/vietnam-agentic-skills --skill advanced-evaluation. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Advanced Evaluation popular?

It lives in huytran1120/vietnam-agentic-skills, which has 9 GitHub stars and 4 forks.

Related skills

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.