Agent Observability Spec
Specify the tracing, metrics, and alerting for an AI agent or LLM feature in production. Use when asked what to log for an LLM app, design agent tracing or spans, define quality and cost monitors, or answer 'how do we know if the agent is misbehaving?'. Produces an observability spec with a trace schema, metric definitions with owners and alert thresholds, sampling and retention policy, and a privacy note for logged content.
Install
/plugin marketplace add mohitagw15856/pm-claude-skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add mohitagw15856/pm-claude-skills --skill agent-observability-specgit clone --depth 1 https://github.com/mohitagw15856/pm-claude-skills.git /tmp/pm-claude-skills
mkdir -p ~/.claude/skills
cp -r /tmp/pm-claude-skills/plugins/pm-agentops/skills/agent-observability-spec ~/.claude/skills/agent-observability-specPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Agent Observability Spec Skill You can't fix what you didn't record. For LLM systems the unit of observability is the *trace* — everything the model saw and did — because behaviour, not uptime, is what fails. This skill specifies what to capture, what to compute from it, and when to page someone. > **Not quite this?** Use `agent-spec` when you are specifying what the agent should do before it is built. ## What This Skill Produces - A **trace schema**: per-request spans and the fields each must carry - **Metric definitions** across health, quality, cost, and behaviour — each with a threshold and owner - A **sampling and retention policy** that keeps cost sane and debugging possible - A **privacy note**: what logged content contains, who can see it, and how long it lives ## Required Inputs Ask for (if not already provided): - **The system's shape** — single LLM call, RAG pipeline, or multi-step tool-using agent - **Traffic volume and cost sensitivity** — full tracing at 10M req/day is a budget decision - **What "misbehaving" means here** — the two or three failure modes that matter most (wrong facts? wrong actions? cost? refusals?) - **Existing observability stack** (Datadog, Langfuse, OTel, homegrown) — spec into it, not around it ## Trace Schema Every request produces one trace; every model call, retrieval, guardrail check, and tool execution is a span. Minimum fields: | Span | Must capture | |---|---|
Details
- Skill name
agent-observability-spec- Repository
- mohitagw15856/pm-claude-skills
- Path
plugins/pm-agentops/skills/agent-observability-spec/SKILL.md- License
- MIT
- Repo updated
- 18 November 2026
- Listed in
- BehiSecc/awesome-claude-skills
FAQ
What does the Agent Observability Spec skill do?
Specify the tracing, metrics, and alerting for an AI agent or LLM feature in production. Use when asked what to log for an LLM app, design agent tracing or spans, define quality and cost monitors, or answer 'how do we know if the agent is misbehaving?'. Produces an observability spec with a trace schema, metric definitions with owners and alert thresholds, sampling and retention policy, and a privacy note for logged content.
How do I install the Agent Observability Spec skill?
Claude Code plugin: /plugin marketplace add mohitagw15856/pm-claude-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Agent Observability Spec popular?
It lives in mohitagw15856/pm-claude-skills, which has 1,573 GitHub stars and 281 forks, and is included in 1 curated skill list.
Related skills
Ab Test Planner
1.6kmohitagw15856/pm-claude-skills
Diseña pruebas A/B estadísticamente rigurosas para características de producto, cambios de UI, flujos de onboarding y experimentos de precios. Úsalo cuando te pidan configurar un experimento, diseñar una prueba A/B, calcular tamaño de muestra o interpretar resultados de pruebas. Produce un plan de prueba completo con hipótesis, definición de variantes, tamaño de muestra, estimación de duración, métricas de seguridad y una guía de interpretación de resultados.
Ab Test Planner
1.6kmohitagw15856/pm-claude-skills
प्रोडक्ट फ़ीचर, UI बदलाव, ऑनबोर्डिंग फ़्लो और प्राइसिंग प्रयोगों के लिए सांख्यिकीय रूप से कठोर A/B टेस्ट डिज़ाइन करें। जब कोई प्रयोग सेट करने, A/B टेस्ट डिज़ाइन करने, सैंपल साइज़ निकालने या टेस्ट के नतीजे समझाने को कहे, तब इसका उपयोग करें। परिकल्पना, वेरिएंट की परिभाषा, सैंपल साइज़, अवधि का अनुमान, गार्डरेल मेट्रिक और नतीजों की व्याख्या की गाइड के साथ एक पूरा टेस्ट प्लान तैयार करता है।
Accessible Travel Planner
1.6kmohitagw15856/pm-claude-skills
Summarise what Accessible Travel Planner does in one line. Use when asked to [trigger phrases the user would say]. Produces [the concrete artifact].
Agent Project File
1.6kmohitagw15856/pm-claude-skills
Set up the project file an AI agent keeps for work that runs over days or weeks: the goal, the definition of done, milestones, the current state, decisions made, open questions, and the exact point to resume from after an interruption. Use when asked to plan a long-running agent task, make my agent work on this over a week, keep state between agent sessions, or hand a multi-day project to an AI. Produces the project file, the update rules, the check-in schedule and the resume checklist.
Agent Negotiation Brief
1.6kmohitagw15856/pm-claude-skills
Brief an AI agent that will negotiate for you with another party's agent or a person (a refund, a price, a meeting time, a contract term): your goal, your opening, your limits, your walk-away point, what it must never reveal, and when it must come back to you. Use when asked to let my agent negotiate, brief my AI for a negotiation, set limits for an agent bargaining on my behalf, or prepare for agent-to-agent negotiation. Produces the brief, the concession ladder, the never-reveal list and the hand-back triggers.
Agent Readiness Audit
1.6kmohitagw15856/pm-claude-skills
Audit whether AI agents can actually use your product — docs, APIs, onboarding, errors, and discoverability, evaluated from a non-human user's perspective. Use when asked if a product is agent-ready, to audit a site or API for AI usability, to prepare for agentic traffic, or when agents keep failing against your product. Produces a scored readiness report with per-surface findings and a prioritised fix list. For optimising a single article for AI citation use aeo-optimizer; for designing the MCP server itself use mcp-server-spec.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.