M365 Agent Evaluator
Use this skill when a user wants to create, run, or analyze evaluation suites for Microsoft 365 Copilot declarative agents with the public @microsoft/m365-copilot-eval CLI. Trigger on intents such as "evaluate my agent", "test my agent", "run my evals", "create eval prompts", "add multi-turn tests", "tune evaluator thresholds", "why is my agent failing", or "set up eval environment variables".
Install
/plugin marketplace add microsoft/skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add microsoft/skills --skill m365-agent-evaluatorgit clone --depth 1 https://github.com/microsoft/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/.github/plugins/microsoft-365-agents-toolkit/skills/m365-agent-evaluator ~/.claude/skills/m365-agent-evaluatorPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# M365 Agent Evaluator Use this skill to help users evaluate Microsoft 365 Copilot declarative agents with `@microsoft/m365-copilot-eval`. The skill designs schema-compatible eval datasets, runs the public preview CLI, analyzes results, and recommends targeted fixes. Default to Microsoft 365 Agents Toolkit (ATK) projects when detected, but do not hard-stop solely because the current directory is not ATK. The CLI can also evaluate deployed agents with an explicit `M365_AGENT_ID` or `--m365-agent-id`. ## Always use this CLI invocation ```powershell npx -y --package @microsoft/m365-copilot-eval@latest runevals ``` Do not recommend the old private `aka.ms` installer, global installs, bare `runevals`, bare `npx runevals`, `--input`, or `--html`. ## Activation workflow 1. Identify the user goal: setup, dataset authoring, running evals, analyzing results, or updating an existing eval suite. 2. Load only the reference needed for the current goal: - `references/workflow.md` for the end-to-end operator workflow and CLI commands. - `references/azure-setup.md` for prerequisites, env files, and secret handling. - `references/eval-templates.md` when creating or editing eval datasets. - `references/pra-framework.md` when deciding what scenarios to generate. - `references/result-analysis.md` after JSON/CSV/HTML results exist. - `references/guardrails.md` before writing files, handling secrets, clearing cache, signing out, or troubleshooting. 3. Detect project shape:
Details
- Skill name
m365-agent-evaluator- Repository
- microsoft/skills
- Path
.github/plugins/microsoft-365-agents-toolkit/skills/m365-agent-evaluator/SKILL.md- License
- MIT
- Repo updated
- 18 November 2026
FAQ
What does the M365 Agent Evaluator skill do?
Use this skill when a user wants to create, run, or analyze evaluation suites for Microsoft 365 Copilot declarative agents with the public @microsoft/m365-copilot-eval CLI. Trigger on intents such as "evaluate my agent", "test my agent", "run my evals", "create eval prompts", "add multi-turn tests", "tune evaluator thresholds", "why is my agent failing", or "set up eval environment variables".
How do I install the M365 Agent Evaluator skill?
Claude Code plugin: /plugin marketplace add microsoft/skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is M365 Agent Evaluator popular?
It lives in microsoft/skills, which has 3,222 GitHub stars and 372 forks, and is included in 2 curated skill lists.
Related skills
Azure AI Textanalytics Py
3.2kmicrosoft/skills
Azure AI Text Analytics SDK for sentiment analysis, entity recognition, key phrases, language detection, PII, and healthcare NLP. Use for natural language processing on text. Triggers: "text analytics", "sentiment analysis", "entity recognition", "key phrase", "PII detection", "TextAnalyticsClient".
Azure AI Anomalydetector Java
3.2kmicrosoft/skills
Build anomaly detection applications with Azure AI Anomaly Detector SDK for Java. Use when implementing univariate/multivariate anomaly detection, time-series analysis, or AI-powered monitoring.
Azure AI Openai Dotnet
3.2kmicrosoft/skills
Azure OpenAI SDK for .NET. Client library for Azure OpenAI and OpenAI services. Use for chat completions, embeddings, image generation, audio transcription, and assistants. Triggers: "Azure OpenAI", "AzureOpenAIClient", "ChatClient", "chat completions .NET", "GPT-4", "embeddings", "DALL-E", "Whisper", "OpenAI .NET".
M365 Agents Dotnet
3.2kmicrosoft/skills
Microsoft 365 Agents SDK for .NET. Build multichannel agents for Teams/M365/Copilot Studio with ASP.NET Core hosting, AgentApplication routing, and MSAL-based auth. Triggers: "Microsoft 365 Agents SDK", "Microsoft.Agents", "AddAgentApplicationOptions", "AgentApplication", "AddAgentAspNetAuthentication", "Copilot Studio client", "IAgentHttpAdapter".
Lieflat Less AI Tone
3.2klarashero3-dotcom/writing-dna-skill
按 SKILL.md 明确列出的规则识别并改写写作中的 AI 痕迹。只能处理清单内的问题;未命中规则的文字必须原样保留,也不能改变文章框架。适用于写作完成后的成稿清理。 Remove AI writing tells using an explicit whitelist of rules; leaves unmatched text untouched.
M365 Agents Py
3.2kmicrosoft/skills
Microsoft 365 Agents SDK for Python. Build multichannel agents for Teams/M365/Copilot Studio with aiohttp hosting, AgentApplication routing, streaming responses, and MSAL-based auth. Triggers: "Microsoft 365 Agents SDK", "microsoft_agents", "AgentApplication", "start_agent_process", "TurnContext", "Copilot Studio client", "CloudAdapter".
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.