Advanced Evaluation
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
Install
npx skills add lucaspmarie-a11y/claude-skills-vault --skill advanced-evaluationgit clone --depth 1 https://github.com/lucaspmarie-a11y/claude-skills-vault.git /tmp/claude-skills-vault
mkdir -p ~/.claude/skills
cp -r /tmp/claude-skills-vault/plugins/antigravity-awesome-skills-claude/skills/advanced-evaluation ~/.claude/skills/advanced-evaluationPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Advanced Evaluation This skill covers production-grade techniques for evaluating LLM outputs using LLMs as judges. It synthesizes research from academic papers, industry practices, and practical implementation experience into actionable patterns for building reliable evaluation systems. **Key insight**: LLM-as-a-Judge is not a single technique but a family of approaches, each suited to different evaluation contexts. Choosing the right approach and mitigating known biases is the core competency this skill develops. ## When to Use Activate this skill when: - Building automated evaluation pipelines for LLM outputs - Comparing multiple model responses to select the best one - Establishing consistent quality standards across evaluation teams - Debugging evaluation systems that show inconsistent results - Designing A/B tests for prompt or model changes - Creating rubrics for human or automated evaluation - Analyzing correlation between automated and human judgments ## Core Concepts ### The Evaluation Taxonomy Evaluation approaches fall into two primary categories with distinct reliability profiles: **Direct Scoring**: A single LLM rates one response on a defined scale. - Best for: Objective criteria (factual accuracy, instruction following, toxicity) - Reliability: Moderate to high for well-defined criteria - Failure mode: Score calibration drift, inconsistent scale interpretation **Pairwise Comparison**: An LLM compares two responses and selects the better one.
Details
- Skill name
advanced-evaluation- Repository
- lucaspmarie-a11y/claude-skills-vault
- Path
plugins/antigravity-awesome-skills-claude/skills/advanced-evaluation/SKILL.md- License
- MIT
- Repo updated
- 6 June 2026
FAQ
What does the Advanced Evaluation skill do?
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
How do I install the Advanced Evaluation skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add lucaspmarie-a11y/claude-skills-vault --skill advanced-evaluation. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Advanced Evaluation popular?
It lives in lucaspmarie-a11y/claude-skills-vault, which has 5 GitHub stars and 0 forks.
Related skills
Apify Influencer Discovery
5lucaspmarie-a11y/claude-skills-vault
Find and evaluate influencers for brand partnerships, verify authenticity, and track collaboration performance across Instagram, Facebook, YouTube, and TikTok.
00 Andruia Consultant
5lucaspmarie-a11y/claude-skills-vault
Arquitecto de Soluciones Principal y Consultor Tecnológico de Andru.ia. Diagnostica y traza la hoja de ruta óptima para proyectos de IA en español.
10 Andruia Skill Smith
5lucaspmarie-a11y/claude-skills-vault
Ingeniero de Sistemas de Andru.ia. Diseña, redacta y despliega nuevas habilidades (skills) dentro del repositorio siguiendo el Estándar de Diamante.
Agent Evaluation
5lucaspmarie-a11y/claude-skills-vault
You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.
AI Ml API Automation
5Engineer1999/A-Curated-List-of-Awesome-Claude-Skills
Automate AI ML API tasks via Advanced MCP. Always search tools first for current schemas.
Agent Tool Builder
5lucaspmarie-a11y/claude-skills-vault
You are an expert in the interface between LLMs and the outside world. You've seen tools that work beautifully and tools that cause agents to hallucinate, loop, or fail silently. The difference is almost always in the design, not the implementation.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.