Eval Driven Dev

Improve AI application with evaluation-driven development. Define eval criteria, instrument the application, build golden datasets, observe and evaluate application runs, analyze results, and produce a concrete action plan for improvements. ALWAYS USE THIS SKILL when the user asks to set up QA, add tests, add evals, evaluate, benchmark, fix wrong behaviors, improve quality, or do quality assurance for any Python project that calls an LLM model.

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add yiouli/pixie-qa --skill eval-driven-dev
Manual install
git clone --depth 1 https://github.com/yiouli/pixie-qa.git /tmp/pixie-qa
mkdir -p ~/.claude/skills
cp -r /tmp/pixie-qa/skills/eval-driven-dev ~/.claude/skills/eval-driven-dev

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Eval-Driven Development for Python LLM Applications

You're building an **automated evaluation pipeline** that tests a Python-based AI application end-to-end — running it the same way a real user would, with real inputs — then scoring the outputs using evaluators and producing pass/fail results via `pixie test`.

**What you're testing is the app itself** — its request handling, context assembly (how it gathers data, builds prompts, manages conversation state), routing, and response formatting. The app uses an LLM, which makes outputs non-deterministic — that's why you use evaluators (LLM-as-judge, similarity scores) instead of `assertEqual` — but the thing under test is the app's code, not the LLM.

During evaluation, the app's own code runs for real — routing, prompt assembly, LLM calls, response formatting — nothing is mocked or stubbed. But the data the app reads from external sources (databases, caches, third-party APIs, voice streams) is replaced with test-specified values via instrumentations. This means each test case controls exactly what data the app sees, while still exercising the full application code path.
Read the full skill on GitHub

Details

Skill name
eval-driven-dev
Path
skills/eval-driven-dev/SKILL.md
License
MIT
Repo updated
22 April 2026

FAQ

What does the Eval Driven Dev skill do?

Improve AI application with evaluation-driven development. Define eval criteria, instrument the application, build golden datasets, observe and evaluate application runs, analyze results, and produce a concrete action plan for improvements. ALWAYS USE THIS SKILL when the user asks to set up QA, add tests, add evals, evaluate, benchmark, fix wrong behaviors, improve quality, or do quality assurance for any Python project that calls an LLM model.

How do I install the Eval Driven Dev skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add yiouli/pixie-qa --skill eval-driven-dev. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Eval Driven Dev popular?

It lives in yiouli/pixie-qa, which has 9 GitHub stars and 0 forks.

Related skills

Fabling

8

gncdev/fabling

Behavioral profile that makes Claude work like the Fable 5 model tier — deep but proportional code investigation, outcome-first communication, high autonomy without permission-asking, faithful outcome reporting, and disciplined turn endings. Use this skill whenever the user asks Claude to behave "like Fable", mentions "fabling", "fable mode" or "fable gibi", complains that Claude is being lazy or superficial, or asks Claude to be more thorough, more autonomous, less chatty, or to stop asking for permission mid-task. Load it once at the start of the session and follow it for every subsequent tu

AI, LLMs & AgentsIn 1 list

Chaos Agent

9

jasnonaz/chaos-agent

Chaos agent for skills — a vigilance-training layer that keeps humans reviewing AI output instead of rubber-stamping it. Consult this skill on EVERY substantive request (code, analysis, writing, SQL, configs, explanations — anything with a checkable answer). 95% of requests get your genuine best work; a deterministic 5%, chosen by hash, get a polished answer containing exactly one subtle, findable flaw. Trigger on all real work; skip only trivial chit-chat.

AI, LLMs & Agents

Jaxs Knife

9

arpasan/jaxs-knife

Bayesian modeling workflow for Stan (CmdStanPy / nutpie[stan]) and JAX (log-density + BlackJAX), sharing ArviZ InferenceData. Encodes the Gelman–Vehtari sequence, Stan geometry (non-centered, Jacobian, generated quantities, diagnose), and JAX-like-Stan log-density patterns that agents skip unprompted. Use when writing or diagnosing .stan programs, CmdStan/CmdStanPy fits, JAX logdensity_fn / BlackJAX NUTS, hierarchical or funnel geometry, partial pooling, measurement error, truncation, censoring, missing data, imperfect assays, divergences, R-hat, ESS, PSIS-LOO, ELPD, stacking, prior predictive

AI, LLMs & Agents

AI Ready

9

viktor-silakov/ai-ready

Analyzes repositories for AI agent development efficiency. Scores 8 aspects (documentation, architecture, testing, type safety, agent instructions, file structure, context optimization, security) with ASCII dashboards. Use when evaluating AI-readiness, preparing codebases for Claude Code, or improving repository structure for AI-assisted development.

AI, LLMs & Agents

Powabase

9

powabase-ai/agent-skills

Use for ANY task building on Powabase, the multi-tenant AI Backend-as-a-Service (projects on *.p.powabase.ai, the /api/* surface). Triggers: RAG & knowledge bases (Sources, document upload/extraction, indexing strategies chunk_embed/full_document/page_index/graph_index/doc2json, retrieval vector/full-text/hybrid/tree search, reranking, query enrichment, multimodal/image retrieval, embeddings, pgvector); agents (ReAct loops, builtin/custom/MCP tools, sessions, streaming, approval/human-in-the-loop); multi-agent orchestrations (supervisor/sequential/parallel); workflows (block graphs, webhooks,

AI, LLMs & Agents

Agentcfg Manage

9

Phoobobo/herdr-agent-config-manager

Manage agent skills and configuration through the agentcfg CLI. Use when the user asks in natural language to inspect skills/MCP/plugins/hooks, centralize or organize scattered skills, install a local skill, link a shared skill into agent harnesses, update installed skills, or safely remove skill links/store copies.

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.