Monitor AI Quality
Monitors AI agent health across quality, cost, performance, and errors. Only use when the user has Amplitude Agent Analytics instrumented in their project. Use when the user asks "how are our AI agents doing", "AI quality check", "agent health", "AI errors", "agent performance", "LLM cost", or wants a proactive health report on their AI/LLM features.
Install
/plugin marketplace add amplitude/mcp-marketplaceThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add amplitude/mcp-marketplace --skill monitor-ai-qualitygit clone --depth 1 https://github.com/amplitude/mcp-marketplace.git /tmp/mcp-marketplace
mkdir -p ~/.claude/skills
cp -r /tmp/mcp-marketplace/plugins/amplitude/skills/monitor-ai-quality ~/.claude/skills/monitor-ai-qualityPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# AI Agent Quality Monitor You are a proactive AI operations advisor that delivers a concise, actionable health report on the user's AI agents. Your goal is to surface quality regressions, error spikes, cost anomalies, and performance degradations — then point to the specific sessions that need attention. ## Instructions ### Phase 1: Get Context and Schema 1. **Get context.** Call `Amplitude:get_amplitude_context` to identify the user's projects and role. 2. **Get AI schema.** Call `Amplitude:get_amplitude_agent_analytics_info` with `view: "schema"` to discover available agent names, tool names, topic models, and rubric definitions. This tells you what's in the data before you query it. 3. **Determine scope.** If the user specifies an agent, time range, or focus area, narrow accordingly. Otherwise default to all agents over the last 7 days. ### Phase 2: Gather the Full Picture Run these in parallel — this is one batch of calls that gives you the complete health snapshot. 1. **Quality + cost + performance overview.** Call `Amplitude:get_amplitude_agent_analytics_info` with `view: "sessions"`, then aggregate quality, cost, latency, sentiment, failures, rubric scores, and error categories by agent from the returned sessions and evaluator results. This gives you the overall and per-agent health snapshot.
Details
- Skill name
monitor-ai-quality- Repository
- amplitude/mcp-marketplace
- Path
plugins/amplitude/skills/monitor-ai-quality/SKILL.md- License
- MIT
- Repo updated
- 17 November 2026
FAQ
What does the Monitor AI Quality skill do?
Monitors AI agent health across quality, cost, performance, and errors. Only use when the user has Amplitude Agent Analytics instrumented in their project. Use when the user asks "how are our AI agents doing", "AI quality check", "agent health", "AI errors", "agent performance", "LLM cost", or wants a proactive health report on their AI/LLM features.
How do I install the Monitor AI Quality skill?
Claude Code plugin: /plugin marketplace add amplitude/mcp-marketplace. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Monitor AI Quality popular?
It lives in amplitude/mcp-marketplace, which has 48 GitHub stars and 12 forks.
Related skills
Add Analytics Instrumentation
48amplitude/mcp-marketplace
End-to-end analytics instrumentation workflow for a PR, branch, file, directory, or feature. Reads the code, discovers what events should be tracked, and produces a concrete instrumentation plan — all in one shot. Use this skill whenever a user wants to add analytics to a PR, asks "instrument this PR", "add tracking to this branch", "what analytics does this file need", "instrument the checkout flow", "run the full instrumentation workflow", or any request that implies going from code changes to a tracking plan. Also trigger when the user gives you a PR link, branch name, file path, or feature
Analyze Account Health
48amplitude/mcp-marketplace
Summarizes B2B account health by analyzing usage patterns, engagement trends, risk signals, and expansion opportunities. Use for customer success reviews, renewal preparation, QBRs, or account prioritization.
Analyze AI Topics
48amplitude/mcp-marketplace
Analyzes what users ask AI agents about and how well each topic is served. Only use when the user has Amplitude Agent Analytics instrumented in their project. Use when the user asks "what are people asking the AI", "top AI topics", "where is the AI struggling", "AI coverage gaps", "what should we improve in our AI", or wants product insights from AI conversation patterns.
Review Agent Insights
48amplitude/mcp-marketplace
Retrieves, synthesizes, and prioritizes all recent AI agent results from Amplitude. Queries every agent type available in get_agent_results, validates freshness, and produces a unified narrative ranked by impact. Use when the user asks "what has the AI found", "show me agent insights", "any AI findings", "what did Amplitude discover", "review AI insights", or wants a digest of everything Amplitude's AI agents have surfaced recently.
Iso 42001
33scytale-labs/GRC-Claude-Skills
Use when the user asks about ISO/IEC 42001 — Artificial Intelligence Management System (AIMS), AI risk assessment, the Annex A AI-specific controls, Annex B implementation guidance, AI governance, AI impact assessment, or using ISO 42001 alongside ISO 27001 for an AI-enabled product. For organizations building, deploying, or governing AI systems.
Taxonomy
48amplitude/mcp-marketplace
Source of truth for event taxonomy generation, data auditing, and governance best practices in Amplitude. Use when an agent needs to create, validate, audit, score, or recommend improvements to event tracking plans, naming conventions, property standards, data quality, or deprecation workflows, including checking a plan against the customer's own standard (Data → Settings naming conventions and AI Context). Covers naming rules, property standards, scoring frameworks, safe metadata operations, deprecation procedures, and AI readiness guidance.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.