Planner Evaluator Harness
Multi-agent harness pattern (planner / generator / evaluator) for long-running application builds, distilled from Anthropic's engineering post on harness design. Use when a build will run for hours or sits beyond what the current model ships reliably solo: covers sprint contracts, skeptical-evaluator calibration, hard-threshold QA grading via browser automation, context resets vs compaction, and how to strip the harness down as models improve.
Install
npx skills add Archive228/lab-skills --skill planner-evaluator-harnessgit clone --depth 1 https://github.com/Archive228/lab-skills.git /tmp/lab-skills
mkdir -p ~/.claude/skills
cp -r /tmp/lab-skills/skills/planner-evaluator-harness ~/.claude/skills/planner-evaluator-harnessPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Planner / Generator / Evaluator Harness for Long Application Builds This skill gives you Anthropic's multi-agent harness pattern for long-running application development: a planner that expands a short prompt into an ambitious spec, a generator that builds in bounded sprints, and a separately-prompted skeptical evaluator that grades the running application through browser automation. The split exists because agents asked to judge their own work confidently praise it even when quality is obviously mediocre; "far more tractable than making a generator critical of its own work" is how the source describes tuning a standalone evaluator to be skeptical. On a retro-game-maker build with Opus 4.5, the harness ($200, 6 hours) shipped a game whose core play loop actually worked — the entity moved and responded to input, with rough edges remaining — where a solo run ($9, 20 minutes) shipped an interface that looked functional but whose core gameplay was silently broken. ## When to use Use the harness when: - A build will run for hours, not minutes, from a short (1–4 sentence) prompt to a full application. - The task sits beyond what the current model completes reliably in a solo run — per the source, the evaluator earns its cost only when the task is past the model's solo-reliability edge. - Quality targets are subjective (e.g., frontend design), where binary self-verification is impossible and a calibrated evaluator makes subjectivity gradable.
Details
- Skill name
planner-evaluator-harness- Repository
- Archive228/lab-skills
- Path
skills/planner-evaluator-harness/SKILL.md- License
- MIT
- Repo updated
- 12 August 2026
FAQ
What does the Planner Evaluator Harness skill do?
Multi-agent harness pattern (planner / generator / evaluator) for long-running application builds, distilled from Anthropic's engineering post on harness design. Use when a build will run for hours or sits beyond what the current model ships reliably solo: covers sprint contracts, skeptical-evaluator calibration, hard-threshold QA grading via browser automation, context resets vs compaction, and how to strip the harness down as models improve.
How do I install the Planner Evaluator Harness skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add Archive228/lab-skills --skill planner-evaluator-harness. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Planner Evaluator Harness popular?
It lives in Archive228/lab-skills, which has 10 GitHub stars and 2 forks.
Related skills
Agent Design Fundamentals
10Archive228/lab-skills
Decision rules for whether to build an LLM agent and how to architect it, distilled from OpenAI's "A Practical Guide to Building Agents": build/no-build criteria, model selection, tool design, single-agent vs multi-agent orchestration (manager pattern vs decentralized handoffs), layered guardrails, and human-in-the-loop triggers. Use when designing, reviewing, or refactoring an agent system — before choosing an architecture, adding more agents, defining tools, or shipping an agent to production without guardrails.
Agent Evals
10Archive228/lab-skills
Build and maintain evaluation suites for AI agents: bootstrap 20-50 tasks from real failures, pick code-based vs model-based vs human graders by agent architecture (coding, conversational, research, computer-use), write unambiguous tasks with reference solutions, and keep suites alive via ownership and transcript reading. Use when creating evals from zero, adding graders or eval cases, debugging suspicious eval scores, or deciding pass@k vs pass^k metrics for an agent product.
Agent Failure Preflight
10Archive228/lab-skills
Pre-deployment failure-mode gate for agentic AI systems, distilled from Microsoft AI Red Team's Taxonomy of Failure Modes in Agentic AI Systems v2.0 (grounded in 12 months of red-team engagements). Run it before shipping or materially changing an agent — adding tools/MCP servers/plugins, enabling persistent memory, adding sub-agents or orchestration, enabling computer use, or expanding permissions — to verify concrete controls against memory poisoning, HitL bypass, goal hijacking, supply-chain compromise, session context contamination, capability disclosure, and multi-agent cascade failures.
Prompt Cache Optimization
10Archive228/lab-skills
Maximize OpenAI prompt-cache hit rate to cut time-to-first-token latency by up to 80% and input token cost by up to 90%: prefix-stability rules, cache-aware prompt layout, prompt_cache_key routing, allowed_tools, Responses API vs Chat Completions, Flex vs Batch, extended 24h caching, and Realtime retention_ratio. Use when designing or reviewing prompt/message layout for any repeated OpenAI API workload, when cached_tokens is low or zero, when TTFT or input spend is high at scale, or when choosing truncation/compaction strategy for long conversations. Scope: models before GPT-5.6 (per the sourc
Phd Application Advisor
5alex-yimingyang/phd-application-advisor
大龄非科班转AI申请海外PhD的全流程顾问。覆盖RA申请、选方向选校、套磁、文书、面试、推荐信、时间规划等所有环节,专门针对30+在职、非CS背景、想转AI方向申博的人群提供策略建议。 当用户提到以下任何场景时使用此skill:大龄读博、非科班申博、转行申PhD、30+申博、在职申博、工作后读博、非传统背景申请PhD、AI方向博士申请、美国博士申请、海外PhD、申博规划、申博咨询,或任何涉及"工作几年后想读博不知道怎么开始"的问题。即使用户没有明确说"非科班",只要描述了类似背景(非CS本科、工作多年、想转AI/ML方向读博),也应触发此skill。
Prompt Engineering Canon
10Archive228/lab-skills
Canonical prompt-engineering operating procedure distilled from Google's 68-page Prompt Engineering whitepaper (Lee Boonstra, Feb 2025): when to use zero/few-shot, system/contextual/role prompting, chain-of-thought, step-back, self-consistency, ToT, ReAct, output-format control, and exact temperature/top-K/top-P starting values. Trigger whenever writing, reviewing, or debugging a prompt for an LLM, choosing sampling configuration, or deciding which prompting technique fits a task.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.