Eval Engineer

Eval Engineer

Eval Engineer turns generic coding agents like Codex and Claude Code into
Galileo-backed eval engineers.

Read the launch post:
Introducing Eval Engineer: Bringing Eval Expertise to Claude and Codex.

The idea is simple: coding agents are good at changing code, but AI apps should
not improve by guesswork. They should improve through evidence. Eval Engineer
gives a coding agent the loop it needs to inspect Galileo traces and metrics,
diagnose the failure, make one bounded change, and verify whether the next run
actually improved.

Quick Installation

Install Eval Engineer into the directory where you will launch the agent. For
Codex, this should be the exact project directory you open with codex:

Project install writes skills into .agents/skills/eval-* for Codex and
.claude/skills/eval-* for Claude Code under that project directory. It also
prepares a minimal .galileo/ workspace without overwriting existing files.

Start a new Claude Code or Codex session from this same project folder after
installing. Do not rely on parent-directory traversal for Codex project skills:
install into the directory that will be the Codex working directory. If Codex is
opened from a child or sibling folder, reinstall there or use a user-scope
install.

Claude Code surfaces the skills as slash commands:

Codex surfaces the same skills as $ mentions:

If the skills do not appear, restart the host agent. For Codex, also confirm the
skills were installed under the project directory you opened:

eval-engineer check validates that files exist at the requested destination.
It cannot prove that an already-running Codex session has discovered the skills;
restart Codex after installing or changing the destination.

The first useful prompt is usually:

In Claude Code, use…