Agent Observability

Use when the user asks about agent observability, tracing agent reasoning, debugging an agent loop, OpenTelemetry GenAI spans for LLM calls, LLM cost tracking per agent, agent audit logs or hallucination/eval monitoring in production. Produces instrumentation config, trace schema, dashboards, alerts and cost attribution. Not for generic service tracing without LLMs (use devops-opentelemetry).

45 stars 6 forksDevOps & Cloud

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add j4flmao/agent-skills --skill agent-observability
Manual install
git clone --depth 1 https://github.com/j4flmao/agent-skills.git /tmp/agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/agent-skills/skills/harness-engineering/agent-observability ~/.claude/skills/agent-observability

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Agent Observability

## Purpose

This skill provides the complete knowledge required to build production-grade
observability systems for AI agent deployments. Unlike traditional software where
execution paths are deterministic and debuggable with standard tools, agent systems
exhibit non-deterministic reasoning loops, branching decision trees, variable-length
tool call chains, and stochastic output variations that demand specialized observability
infrastructure.

The skill covers every dimension of agent observability: capturing and visualizing
reasoning traces that show *why* an agent made each decision, integrating OpenTelemetry
to instrument every LLM call and tool invocation, propagating distributed trace context
across multi-agent chains, maintaining tamper-evident decision audit logs, profiling
performance bottlenecks in agent pipelines, detecting anomalous behavior patterns that
indicate drift or failure, tracking costs across model providers and optimizing token
usage, and analyzing latency to identify and eliminate bottlenecks in the agent execution
critical path.

## Core Principles

1. **Trace Everything, Sample Intelligently**: Every LLM call, tool invocation, and
   decision point must be instrumentable. In production, use head-based and tail-based
   sampling to control volume while guaranteeing capture of errors and anomalies.

2. **Reasoning is the Primary Signal**: Traditional metrics (latency, throughput, errors)
Read the full skill on GitHub

Details

Skill name
agent-observability
Path
skills/harness-engineering/agent-observability/SKILL.md
License
MIT
Repo updated
17 November 2026

FAQ

What does the Agent Observability skill do?

Use when the user asks about agent observability, tracing agent reasoning, debugging an agent loop, OpenTelemetry GenAI spans for LLM calls, LLM cost tracking per agent, agent audit logs or hallucination/eval monitoring in production. Produces instrumentation config, trace schema, dashboards, alerts and cost attribution. Not for generic service tracing without LLMs (use devops-opentelemetry).

How do I install the Agent Observability skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add j4flmao/agent-skills --skill agent-observability. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Agent Observability popular?

It lives in j4flmao/agent-skills, which has 45 GitHub stars and 6 forks.

Related skills

AI Agent Architectures

45

j4flmao/agent-skills

Use when the user asks to build an AI agent, design an agent that calls tools and keeps memory, write an agent system prompt or pick an agent framework (Claude Agent SDK, OpenAI Agents SDK, LangGraph). Produces an architecture brief covering loop, tools, memory, guardrails and evals, routing depth to child skills. Not for workflow patterns (use ai-agentic-workflows).

AI, LLMs & Agents

AI Agent Core Loop

45

j4flmao/agent-skills

Use when the user asks about the agent loop, ReAct vs plan-and-solve, re-planning, Reflexion, an agent stuck in a loop, max iterations or termination conditions. Produces a loop spec or patch with state, success predicate, caps and no-progress detection. Not for overall agent design (use ai-agent-architectures) or tool schemas (use tool-calling-principles).

AI, LLMs & Agents

AI Agent Memory Systems

45

j4flmao/agent-skills

Use when the user asks about agent memory, long-term memory across sessions, remembering user preferences, context window full, conversation summarization, episodic or semantic memory. Produces memory tiers, a store schema and write/retrieve/expiry policies with per-user isolation. Not for document RAG (use llm-rag-retrieval-internals) or vector DB operations (use data-vector-databases).

AI, LLMs & Agents

Alibaba Cloud

45

j4flmao/agent-skills

Use when the user asks about Alibaba Cloud (Aliyun), ECS, ACK, OSS, SLB/ALB, ApsaraDB RDS, PolarDB, RAM policies, Function Compute, or the Terraform alicloud provider. Produces Alibaba Cloud architectures, RAM and network configs and IaC. Not for AWS (use aws), Azure (use devops-azure) or GCP (use devops-gcp).

DevOps & Cloud

AI Canary Deployment

39

lionelsimai/claude-skills-collection

Design canary deployment strategies for AI. TRIGGERS - Use when user needs help with ai-canary-deployment related tasks.

DevOps & Cloud

Ansible

45

j4flmao/agent-skills

Use when the user asks about Ansible playbooks, roles, inventory, ansible-playbook, ansible-galaxy collections, Jinja2 templates, ansible-vault, AWX or Ansible Automation Platform, or idempotent configuration management. Produces idempotent playbooks, roles and CI checks (ansible-lint, Molecule). Not for provisioning cloud resources (use terraform).

DevOps & Cloud

Browse the most popular Claude skills, all devops & cloud skills, or write your own with the skills generator.