Eval Driven Dev
Improve AI application with evaluation-driven development. Define eval criteria, instrument the application, build golden datasets, observe and evaluate application runs, analyze results, and produce a concrete action plan for improvements. ALWAYS USE THIS SKILL when the user asks to set up QA, add tests, add evals, evaluate, benchmark, fix wrong behaviors, improve quality, or do quality assurance for any Python project that calls an LLM model.
Install
npx skills add yiouli/pixie-qa --skill eval-driven-devgit clone --depth 1 https://github.com/yiouli/pixie-qa.git /tmp/pixie-qa
mkdir -p ~/.claude/skills
cp -r /tmp/pixie-qa/skills/eval-driven-dev ~/.claude/skills/eval-driven-devPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Eval-Driven Development for Python LLM Applications You're building an **automated evaluation pipeline** that tests a Python-based AI application end-to-end — running it the same way a real user would, with real inputs — then scoring the outputs using evaluators and producing pass/fail results via `pixie test`. **What you're testing is the app itself** — its request handling, context assembly (how it gathers data, builds prompts, manages conversation state), routing, and response formatting. The app uses an LLM, which makes outputs non-deterministic — that's why you use evaluators (LLM-as-judge, similarity scores) instead of `assertEqual` — but the thing under test is the app's code, not the LLM. During evaluation, the app's own code runs for real — routing, prompt assembly, LLM calls, response formatting — nothing is mocked or stubbed. But the data the app reads from external sources (databases, caches, third-party APIs, voice streams) is replaced with test-specified values via instrumentations. This means each test case controls exactly what data the app sees, while still exercising the full application code path.
Details
- Skill name
eval-driven-dev- Repository
- yiouli/pixie-qa
- Path
skills/eval-driven-dev/SKILL.md- License
- MIT
- Repo updated
- 22 April 2026
FAQ
What does the Eval Driven Dev skill do?
Improve AI application with evaluation-driven development. Define eval criteria, instrument the application, build golden datasets, observe and evaluate application runs, analyze results, and produce a concrete action plan for improvements. ALWAYS USE THIS SKILL when the user asks to set up QA, add tests, add evals, evaluate, benchmark, fix wrong behaviors, improve quality, or do quality assurance for any Python project that calls an LLM model.
How do I install the Eval Driven Dev skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add yiouli/pixie-qa --skill eval-driven-dev. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Eval Driven Dev popular?
It lives in yiouli/pixie-qa, which has 9 GitHub stars and 0 forks.
Related skills
Fabling
8gncdev/fabling
Behavioral profile that makes Claude work like the Fable 5 model tier — deep but proportional code investigation, outcome-first communication, high autonomy without permission-asking, faithful outcome reporting, and disciplined turn endings. Use this skill whenever the user asks Claude to behave "like Fable", mentions "fabling", "fable mode" or "fable gibi", complains that Claude is being lazy or superficial, or asks Claude to be more thorough, more autonomous, less chatty, or to stop asking for permission mid-task. Load it once at the start of the session and follow it for every subsequent tu
Chaos Agent
9jasnonaz/chaos-agent
Chaos agent for skills — a vigilance-training layer that keeps humans reviewing AI output instead of rubber-stamping it. Consult this skill on EVERY substantive request (code, analysis, writing, SQL, configs, explanations — anything with a checkable answer). 95% of requests get your genuine best work; a deterministic 5%, chosen by hash, get a polished answer containing exactly one subtle, findable flaw. Trigger on all real work; skip only trivial chit-chat.
Jaxs Knife
9arpasan/jaxs-knife
Bayesian modeling workflow for Stan (CmdStanPy / nutpie[stan]) and JAX (log-density + BlackJAX), sharing ArviZ InferenceData. Encodes the Gelman–Vehtari sequence, Stan geometry (non-centered, Jacobian, generated quantities, diagnose), and JAX-like-Stan log-density patterns that agents skip unprompted. Use when writing or diagnosing .stan programs, CmdStan/CmdStanPy fits, JAX logdensity_fn / BlackJAX NUTS, hierarchical or funnel geometry, partial pooling, measurement error, truncation, censoring, missing data, imperfect assays, divergences, R-hat, ESS, PSIS-LOO, ELPD, stacking, prior predictive
AI Ready
9viktor-silakov/ai-ready
Analyzes repositories for AI agent development efficiency. Scores 8 aspects (documentation, architecture, testing, type safety, agent instructions, file structure, context optimization, security) with ASCII dashboards. Use when evaluating AI-readiness, preparing codebases for Claude Code, or improving repository structure for AI-assisted development.
Powabase
9powabase-ai/agent-skills
Use for ANY task building on Powabase, the multi-tenant AI Backend-as-a-Service (projects on *.p.powabase.ai, the /api/* surface). Triggers: RAG & knowledge bases (Sources, document upload/extraction, indexing strategies chunk_embed/full_document/page_index/graph_index/doc2json, retrieval vector/full-text/hybrid/tree search, reranking, query enrichment, multimodal/image retrieval, embeddings, pgvector); agents (ReAct loops, builtin/custom/MCP tools, sessions, streaming, approval/human-in-the-loop); multi-agent orchestrations (supervisor/sequential/parallel); workflows (block graphs, webhooks,
Agentcfg Manage
9Phoobobo/herdr-agent-config-manager
Manage agent skills and configuration through the agentcfg CLI. Use when the user asks in natural language to inspect skills/MCP/plugins/hooks, centralize or organize scattered skills, install a local skill, link a shared skill into agent harnesses, update installed skills, or safely remove skill links/store copies.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.