Dspy Evaluation Suite
Use for evaluating DSPy programs with Evaluate, answer_exact_match, SemanticF1, custom metrics, baselines, and program comparisons.
Install
/plugin marketplace add OmidZamani/dspy-skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add OmidZamani/dspy-skills --skill dspy-evaluation-suitegit clone --depth 1 https://github.com/OmidZamani/dspy-skills.git /tmp/dspy-skills
mkdir -p ~/.claude/skills
cp -r /tmp/dspy-skills/skills/dspy-evaluation-suite ~/.claude/skills/dspy-evaluation-suitePersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# DSPy Evaluation Suite
## Goal
Systematically evaluate DSPy programs using built-in and custom metrics with parallel execution.
## When to Use
- Measuring program performance before/after optimization
- Comparing different program variants
- Establishing baselines
- Validating production readiness
## Related Skills
- Use with any optimizer: [dspy-bootstrap-fewshot](../dspy-bootstrap-fewshot/SKILL.md), [dspy-miprov2-optimizer](../dspy-miprov2-optimizer/SKILL.md), [dspy-gepa-reflective](../dspy-gepa-reflective/SKILL.md)
- Evaluate RAG pipelines: [dspy-rag-pipeline](../dspy-rag-pipeline/SKILL.md)
## Inputs
| Input | Type | Description |
|-------|------|-------------|
| `program` | `dspy.Module` | Program to evaluate |
| `devset` | `list[dspy.Example]` | Evaluation examples |
| `metric` | `callable` | Scoring function |
| `num_threads` | `int` | Parallel threads |
## Outputs
| Output | Type | Description |
|--------|------|-------------|
| `score` | `float` | Average metric score |
| `results` | `list` | Per-example results |
## Workflow
### Phase 1: Setup Evaluator
```python
from dspy.evaluate import Evaluate
evaluator = Evaluate(
devset=devset,
metric=my_metric,
num_threads=8,
display_progress=True
)
```
### Phase 2: Run EvaluationDetails
- Skill name
dspy-evaluation-suite- Repository
- OmidZamani/dspy-skills
- Path
skills/dspy-evaluation-suite/SKILL.md- License
- MIT
- Repo updated
- 23 June 2026
- Listed in
- heilcheng/awesome-agent-skills
FAQ
What does the Dspy Evaluation Suite skill do?
Use for evaluating DSPy programs with Evaluate, answer_exact_match, SemanticF1, custom metrics, baselines, and program comparisons.
How do I install the Dspy Evaluation Suite skill?
Claude Code plugin: /plugin marketplace add OmidZamani/dspy-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Dspy Evaluation Suite popular?
It lives in OmidZamani/dspy-skills, which has 125 GitHub stars and 13 forks, and is included in 1 curated skill list.
Related skills
Dspy Adapters Multimodal
125OmidZamani/dspy-skills
Use for DSPy adapter selection, JSONAdapter, XMLAdapter, ChatAdapter, native function calling, structured outputs, and multimodal inputs like dspy.Image or dspy.Audio.
Dspy Advanced Module Composition
125OmidZamani/dspy-skills
Use for composing DSPy modules with Ensemble, MultiChainComparison, ensemble voting, sequential pipelines, and multi-program workflows.
Dspy Better Together
125OmidZamani/dspy-skills
Use for BetterTogether, prompt plus weight optimization, fine-tuning sequences, and strategy chains like p -> w -> p.
Dspy Miprov2 Optimizer
125OmidZamani/dspy-skills
Use for MIPROv2, Bayesian optimization, instruction and demo tuning, and high-performance DSPy program optimization.
Code Cleanup Helper
89Hao0321/claude-skill-code-cleanup
Read-only code/architecture/repo/release/Skill audit;含 sync、資安、資產、Token、GitHub 更新/回滾/淘汰;R&D measurement provider。
Dspy Reasoning Modules
125OmidZamani/dspy-skills
Use for DSPy reasoning modules including RLM, ProgramOfThought, CodeAct, Parallel, sandboxed execution, and long-context workflows.
Browse the most popular Claude skills, all software development skills, or write your own with the skills generator.