Copilot Studio Agent Test

Test Microsoft Copilot Studio agents programmatically against a set of question/answer pairs, without driving a browser. Use when the user wants to evaluate, test, validate, benchmark or regression-test a Copilot Studio agent; run an eval set or test set against an agent; check whether an agent answers correctly from its knowledge bases; verify agent grounding or citations; or ask a Copilot Studio agent a question from the command line. Triggers include 'test my Copilot Studio agent', 'run an eval against the agent', 'validate agent answers', 'check the agent knowledge base', 'ask the agent',

110 stars 122 forksTesting & QAOfficial

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add microsoft/cat-agent-skills --skill copilot-studio-agent-test
Manual install
git clone --depth 1 https://github.com/microsoft/cat-agent-skills.git /tmp/cat-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/cat-agent-skills/submissions/copilot-studio-agent-test ~/.claude/skills/copilot-studio-agent-test

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Copilot Studio agent testing

Runs questions against a Copilot Studio agent through the Microsoft 365 Agents SDK
(`CopilotStudio.Copilots.Invoke`), captures answers and citations, and writes results to
CSV and JSON for review.

Each question runs in a **fresh conversation** — the programmatic equivalent of clicking
*New chat* — so no answer benefits from context set by a previous one.

> **Requires Node.js 22 or later.** There is no bundled executable — the tool is a single
> JavaScript file, so nothing unsigned is executed and endpoint policy has nothing to block.
>
> Invoke it from this skill's directory. **Prefer the `node` form** — it is identical on
> Windows, macOS and Linux, and works in every shell:
>
> ```
> node scripts/cs-agent-test.cjs <command>
> ```
>
> Convenience launchers exist if you would rather not type `node`:
> `.\scripts\cs-agent-test.cmd` on Windows, `sh scripts/cs-agent-test.sh` elsewhere.
>
> Examples below use the `node` form, so they can be copied as-is on any platform.
>
> **Check Node first.** If `node --version` fails or reports below v22, stop and tell the user
> to install Node.js 22+ from https://nodejs.org — every command will fail until they do.

## What this tool does and does not do

**Does:** send questions, capture answers, extract citations, detect escalation, flag empty
turns, check optional required substrings, convert Word Q&A documents into test sets, and
assemble a pass/fail report from verdicts it is given.
Read the full skill on GitHub

Details

Skill name
copilot-studio-agent-test
Path
submissions/copilot-studio-agent-test/SKILL.md
License
MIT
Repo updated
15 November 2026

FAQ

What does the Copilot Studio Agent Test skill do?

Test Microsoft Copilot Studio agents programmatically against a set of question/answer pairs, without driving a browser. Use when the user wants to evaluate, test, validate, benchmark or regression-test a Copilot Studio agent; run an eval set or test set against an agent; check whether an agent answers correctly from its knowledge bases; verify agent grounding or citations; or ask a Copilot Studio agent a question from the command line. Triggers include 'test my Copilot Studio agent', 'run an eval against the agent', 'validate agent answers', 'check the agent knowledge base', 'ask the agent',

How do I install the Copilot Studio Agent Test skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add microsoft/cat-agent-skills --skill copilot-studio-agent-test. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Copilot Studio Agent Test popular?

It lives in microsoft/cat-agent-skills, which has 110 GitHub stars and 122 forks.

Related skills

3 Statements

110

microsoft/cat-agent-skills

Completes existing income statement, balance sheet and cash flow templates while preserving layout, grounding historical inputs and linking forecast formulas with reconciliation checks. Use for "fill in a 3-statement template", "complete an IS/BS/CF framework", "populate my integrated financial model", "link statements in this template", or "finish the forecasts in my three-statement workbook". Do NOT use for blank-slate models or general workbook edits (use xlsx), DCF valuation (use dcf-valuation), or buyout modeling (use lbo-model).

AI, LLMs & Agents122

Accessibility Pass

110

microsoft/cat-agent-skills

Use this skill whenever the user asks to check, review, or fix the accessibility of a PowerPoint deck, Word document, HTML page, or Markdown file, and before handing over any deck or document this agent just generated, so it does not ship with missing alt text, untitled slides, or unreadable colour contrast.

Frontend & Design122

Accessible Learning Materials

110

microsoft/cat-agent-skills

Maintains instructionally equivalent, accessible variants of a set of learning materials — lessons, readings, slides, and handouts — with parity checks against the original learning goals and recorded approvals. Use when the user asks to "make these materials accessible", "audit this lesson set for accessibility", "create an accessible version of this reading", "check reading level and alt text", or "keep my accessible variants in sync". Do NOT use for individual student accommodation or support-plan decisions (IEP, EHCP, or local equivalent), and do NOT use for generic document formatting (us

Learning & Education122

Idea Refiner

110

microsoft/cat-agent-skills

Refine the user's existing plan, decision, or draft through relentless refinement. Use when the user wants to sharpen or stress-test their thinking.

Testing & QA122

Classic Text Adventure

110

microsoft/cat-agent-skills

Play and operate a deterministic, resumable Colossal Cave text adventure with verbatim game output and an OOC escape. Use when a user asks to start, continue, inspect, reset, or smoke-test the Classic Text Adventure; treat unprefixed messages as game commands while a session is active.

Testing & QA122

Promo Check

110

microsoft/cat-agent-skills

Presents the promotion eligibility already computed by policy_resolve - the three-part test (window AND cluster AND SKU) with the failing leg and exact dates named. Use for "is this on promo", "does this customer get the deal", "is the discount still running", "why isn't the promo applying", "did the offer end". Do NOT use to resolve the clause or compute eligibility - use policy-retrieve instead (it populates promo_check); not for return or price-match questions - use policy-retrieve; not for product comparisons - use product-compare; not to draft the floor answer - use answer-draft.

Testing & QA122

Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.