Agent Improvement Loop

Operate a trace-driven improvement loop for a deployed agent: capture execution traces, convert human and model feedback into Promptfoo evals, gate the current harness against them, rank harness changes with HALO, and hand off a Codex-ready change set. Use when an agent is live and you need a repeatable cadence for diagnosing failures and shipping harness fixes — not for pre-launch prompt tinkering without traces.

10 stars 2 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add Archive228/lab-skills --skill agent-improvement-loop
Manual install
git clone --depth 1 https://github.com/Archive228/lab-skills.git /tmp/lab-skills
mkdir -p ~/.claude/skills
cp -r /tmp/lab-skills/skills/agent-improvement-loop ~/.claude/skills/agent-improvement-loop

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Agent Improvement Loop: Traces → Feedback → Evals → Ranked Harness Changes → Codegen

This skill gives you the operational flywheel from OpenAI's Agents SDK cookbook for continuously improving a deployed agent: run the agent to produce traces, attach human and model feedback to specific traces, compile that feedback into reusable Promptfoo evals, score the current harness against them as a validation gate, run a HALO optimization pass to rank the next harness changes with evidence, and write a handoff file a codegen agent (Codex) can implement. Each stage keeps what the previous one learned: traces record the agent's actual behavior, feedback pins down which behaviors mattered, evals turn those expectations into checks you can re-run, and the ranked change set gives the codegen agent something concrete to build.

## When to use

Use this loop when:
- An agent is running (or runnable) end-to-end and you have, or can generate, real execution traces.
- You are deciding *which* harness change to make next and want the decision ranked by trace evidence rather than intuition.
- Reviewers or users have given feedback on specific runs and you need to convert it into regression checks before changing anything.
- You are setting up a recurring improve-and-ship cadence for an agent already in production.

Do NOT use this loop when:
- There is no runnable agent yet — the loop needs traces as its raw material.
Read the full skill on GitHub

Details

Skill name
agent-improvement-loop
Path
skills/agent-improvement-loop/SKILL.md
License
MIT
Repo updated
12 August 2026

FAQ

What does the Agent Improvement Loop skill do?

Operate a trace-driven improvement loop for a deployed agent: capture execution traces, convert human and model feedback into Promptfoo evals, gate the current harness against them, rank harness changes with HALO, and hand off a Codex-ready change set. Use when an agent is live and you need a repeatable cadence for diagnosing failures and shipping harness fixes — not for pre-launch prompt tinkering without traces.

How do I install the Agent Improvement Loop skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add Archive228/lab-skills --skill agent-improvement-loop. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Agent Improvement Loop popular?

It lives in Archive228/lab-skills, which has 10 GitHub stars and 2 forks.

Related skills

Agent Design Fundamentals

10

Archive228/lab-skills

Decision rules for whether to build an LLM agent and how to architect it, distilled from OpenAI's "A Practical Guide to Building Agents": build/no-build criteria, model selection, tool design, single-agent vs multi-agent orchestration (manager pattern vs decentralized handoffs), layered guardrails, and human-in-the-loop triggers. Use when designing, reviewing, or refactoring an agent system — before choosing an architecture, adding more agents, defining tools, or shipping an agent to production without guardrails.

AI, LLMs & Agents

Agent Evals

10

Archive228/lab-skills

Build and maintain evaluation suites for AI agents: bootstrap 20-50 tasks from real failures, pick code-based vs model-based vs human graders by agent architecture (coding, conversational, research, computer-use), write unambiguous tasks with reference solutions, and keep suites alive via ownership and transcript reading. Use when creating evals from zero, adding graders or eval cases, debugging suspicious eval scores, or deciding pass@k vs pass^k metrics for an agent product.

AI, LLMs & Agents

Agent Failure Preflight

10

Archive228/lab-skills

Pre-deployment failure-mode gate for agentic AI systems, distilled from Microsoft AI Red Team's Taxonomy of Failure Modes in Agentic AI Systems v2.0 (grounded in 12 months of red-team engagements). Run it before shipping or materially changing an agent — adding tools/MCP servers/plugins, enabling persistent memory, adding sub-agents or orchestration, enabling computer use, or expanding permissions — to verify concrete controls against memory poisoning, HitL bypass, goal hijacking, supply-chain compromise, session context contamination, capability disclosure, and multi-agent cascade failures.

AI, LLMs & Agents

Agent Sandboxing

10

Archive228/lab-skills

Containment architecture for agent products that execute code, browse, or call tools, distilled from Anthropic's "How we contain Claude across products". Use when designing or reviewing sandboxing, isolation, permission systems, egress controls, or untrusted-content handling for an agent, or when choosing between container, OS-sandbox, and VM isolation patterns; provides a threat-model checklist covering user misuse, model misbehavior, and external attacks.

AI, LLMs & Agents

Agent Sessions

9

vladmdgolam/agent-skills

Search, list, and resume AI agent sessions across Claude Code, Codex CLI, Gemini CLI, opencode, Hermes Agent, and Cline CLI. Use when the user asks to find a past conversation, search session history, resume a previous session, list recent agent activity, or check what was discussed in a prior session. Also use when asked to "find that conversation where I...", "resume my last codex session", "read latest convo", "read last messages", "what did I work on yesterday", "search my claude history for X", or gives an agent session id such as `ses_...`, a UUID, or a Cline id like `1782988565588_qqrxi

AI, LLMs & Agents

Agentic Coding Discipline

10

Archive228/lab-skills

Working discipline for agentic coding sessions, distilled from Claude Code's official best-practices doc: close the loop with runnable verification (tests, /goal, Stop hooks, adversarial review subagents), separate explore/plan/code phases, keep CLAUDE.md lean, choose permissions/sandboxing deliberately, manage context aggressively (/clear, /compact, checkpoints), and scale with fan-out and agent teams. Use at the start of any non-trivial coding-agent session, when configuring an agent environment, or when a session shows drift, bloat, or unverified "done" claims.

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.