QA Agent Testing

QA harness for agentic systems: scenario suites, determinism controls, tool sandboxing, scoring rubrics, and regression protocols covering success, safety, latency, and cost.

8 stars 3 forksTesting & QA Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add Freddy-CHEN-ux/00_claude_skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add Freddy-CHEN-ux/00_claude_skills --skill qa-agent-testing
Manual install
git clone --depth 1 https://github.com/Freddy-CHEN-ux/00_claude_skills.git /tmp/00_claude_skills
mkdir -p ~/.claude/skills
cp -r /tmp/00_claude_skills/skills/qa-agent-testing ~/.claude/skills/qa-agent-testing

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# QA Agent Testing (Dec 2025)

Systematic quality assurance framework for LLM agents and personas.

## Core QA (Default)

### What “Agent Testing” Means

- Validate a multi-step system that may use tools, memory, and external data.
- Expect non-determinism; treat variance as a reliability signal, not an excuse.

### Determinism and Flake Control

- Control inputs: pinned prompts/config, fixtures, stable tool responses, frozen time/timezone where possible.
- Control sampling: fixed seeds/temperatures where supported; log model/config versions.
- Record tool traces: tool name, args, outputs, latency, errors, and retries.

### Evaluation Dimensions (Score What Matters)

- Task success: correct outcome and constraints met.
- Safety/policy: correct refusals and safe alternatives.
- Reliability: stability across reruns and small prompt changes.
- Latency and cost: budgets per task and per suite [Inference].
- Debuggability: failures produce evidence (tool logs, traces, intermediate artifacts).

### CI Economics

- PR gate: small, high-signal smoke eval suite.
- Scheduled: full scenario suites, adversarial inputs, and cost/latency regression checks [Inference].

### Do / Avoid

Do:
- Use objective oracles (schema validation, golden traces, deterministic tool mocks) in addition to human review.
- Quarantine flaky evals with owners and expiry, just like flaky tests in CI.

Avoid:
- Evaluating only “happy prompts” with no tool failures and no adversarial inputs.
Read the full skill on GitHub

Details

Skill name
qa-agent-testing
Path
skills/qa-agent-testing/SKILL.md
Repo updated
10 January 2026

FAQ

What does the QA Agent Testing skill do?

QA harness for agentic systems: scenario suites, determinism controls, tool sandboxing, scoring rubrics, and regression protocols covering success, safety, latency, and cost.

How do I install the QA Agent Testing skill?

Claude Code plugin: /plugin marketplace add Freddy-CHEN-ux/00_claude_skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is QA Agent Testing popular?

It lives in Freddy-CHEN-ux/00_claude_skills, which has 8 GitHub stars and 3 forks.

Related skills

Add Integration

8

Freddy-CHEN-ux/00_claude_skills

Build a new API integration for Nexus. Load when user mentions "add integration", "new integration", "integrate with", "connect to [service]", or "build [service] integration". Interactive workflow that discovers API endpoints, plans the integration, and creates a project for implementation.

Software Development

AI Agents

8

Freddy-CHEN-ux/00_claude_skills

Production-grade AI agent patterns with MCP integration, agentic RAG, handoff orchestration, multi-layer guardrails, and observability (modern best practices)

AI, LLMs & Agents

AI LLM

8

Freddy-CHEN-ux/00_claude_skills

Production LLM engineering skill. Covers strategy selection (prompting vs RAG vs fine-tuning), dataset design, PEFT/LoRA, evaluation workflows, deployment handoff to inference serving, and lifecycle operations with cost/safety controls.

AI, LLMs & Agents

QA API Testing Contracts

8

Freddy-CHEN-ux/00_claude_skills

API testing and contract validation. Design and execute schema validation, contract tests, negative testing, and change safety for REST, GraphQL, and gRPC APIs. Use when you need API test plans, contract testing, or CI quality gates.

Testing & QA

Pm Analyst

8

Freddy-CHEN-ux/00_claude_skills

End-to-end PM workflow - from data analysis to pragmatic Linear tickets. Analyzes user behavior, identifies opportunities, creates specs, and audits for simplicity.

Testing & QA

QA Handoff

5

vnovakovits/claude-skills

Update a ticket with what shipped and tell QA what to test, in lean, empathetic, user-journey language. Verifies what is actually merged, released and deployed, maps each change to the ticket it fixes, writes a short status (in production / open / left, by owner) and a numbered list of user journeys to test with expected outcomes and what to report. Use when a change is merged or deployed and someone asks "what should QA test", "update the ticket", "post what we merged", "is this live", or wants a tester note, release note or status comment for any tracker (Linear, Jira, GitHub Issues).

Testing & QA

Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.