Dspy Evaluation Harness
Build DSPy evaluation harnesses with rich-feedback metrics that are essential for GEPA optimization. Use when writing a metric function, calling dspy.Evaluate, splitting dev/val sets, debugging "why is my optimizer not improving?", or designing CI-ready DSPy eval suites.
Install
/plugin marketplace add intertwine/dspy-agent-skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add intertwine/dspy-agent-skills --skill dspy-evaluation-harnessgit clone --depth 1 https://github.com/intertwine/dspy-agent-skills.git /tmp/dspy-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/dspy-agent-skills/skills/dspy-evaluation-harness ~/.claude/skills/dspy-evaluation-harnessPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# DSPy Evaluation Harness (3.2.x) The metric is usually more important than the program. For `dspy.GEPA` especially, the quality of **textual feedback** in your metric determines whether optimization converges. ## Two rules 1. **Return a `dspy.Prediction(score=..., feedback=...)`, not a dict.** `dspy.Evaluate`'s parallel executor aggregates scores via sum, which breaks on dict outputs (`TypeError: unsupported operand type(s) for +: 'int' and 'dict'`). `dspy.Prediction` supports `__float__`/`__add__` and is what GEPA's adapter natively unwraps. A bare float still works for pure `dspy.Evaluate` scoring, but GEPA needs the score+feedback pair. 2. **Separate valset.** Never optimize and evaluate on the same examples. Optimizers overfit fast. ## Canonical rich-feedback metric
Details
- Skill name
dspy-evaluation-harness- Repository
- intertwine/dspy-agent-skills
- Path
skills/dspy-evaluation-harness/SKILL.md- License
- MIT
- Repo updated
- 6 September 2026
FAQ
What does the Dspy Evaluation Harness skill do?
Build DSPy evaluation harnesses with rich-feedback metrics that are essential for GEPA optimization. Use when writing a metric function, calling dspy.Evaluate, splitting dev/val sets, debugging "why is my optimizer not improving?", or designing CI-ready DSPy eval suites.
How do I install the Dspy Evaluation Harness skill?
Claude Code plugin: /plugin marketplace add intertwine/dspy-agent-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Dspy Evaluation Harness popular?
It lives in intertwine/dspy-agent-skills, which has 280 GitHub stars and 24 forks.
Related skills
Dspy Advanced Workflow
280intertwine/dspy-agent-skills
Build DSPy 3.2.x programs through spec, program, metric and baseline; extend to optimization and export when requested and justified by task budget. Orchestrates the other four DSPy skills (dspy-fundamentals, dspy-evaluation-harness, dspy-gepa-optimizer, dspy-rlm-module) in the correct order. Use for greenfield DSPy builds; prototypes may stop at a validated baseline.
Dspy Fundamentals
280intertwine/dspy-agent-skills
Write idiomatic DSPy 3.2.x programs — typed Signatures, dspy.Module subclasses, Predict/ChainOfThought/ReAct/ProgramOfThought, and save/load. Use this when starting any new DSPy project or when fixing non-idiomatic DSPy code (hard-coded prompts, ad-hoc string templates, untyped outputs, non-serializable classes).
Dspy Gepa Optimizer
280intertwine/dspy-agent-skills
Optimize DSPy programs with dspy.GEPA — a reflective/evolutionary optimizer to consider against task-specific baselines within an authorized evaluation budget. Use when the user says optimize, compile, GEPA, reflective optimization, or "make this program better" and a DSPy program + metric + trainset exist.
Internal Comms
193MavCui/Agent-Skills-Hunter
A set of resources to help me write all kinds of internal communications, using the formats that my company likes to use. Claude should use this skill whenever asked to write some sort of internal communications (status reports, leadership updates, 3P updates, company newsletters, FAQs, incident reports, project updates, etc.).
Code Documentation Doc Generate
514Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI
You are a documentation expert specializing in creating comprehensive, maintainable documentation from code. Generate API docs, architecture diagrams, user guides, and technical references using AI-powered analysis and industry best practices.
Content Creation
492cosmicstack-labs/mercury-agent-skills
Comprehensive guide to strategic content creation covering content strategy, writing frameworks, optimization techniques, distribution channels, performance measurement, and AI-assisted content workflows. Designed for marketers and content professionals building audience-focused content programs.
Browse the most popular Claude skills, all writing & content skills, or write your own with the skills generator.