Model Markov Chain

Build and analyze discrete or continuous Markov chains including transition matrix construction, state classification, stationary distribution computation, and mean first passage times. Use when modeling a memoryless system with observed transition counts or rates, computing long-run steady-state probabilities, determining expected hitting times or absorption probabilities, classifying states as transient or recurrent, or building a foundation for hidden Markov models or reinforcement learning MDPs.

45 stars 4 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add pjt222/agent-almanac --skill model-markov-chain
Manual install
git clone --depth 1 https://github.com/pjt222/agent-almanac.git /tmp/agent-almanac
mkdir -p ~/.claude/skills
cp -r /tmp/agent-almanac/i18n/caveman-lite/skills/model-markov-chain ~/.claude/skills/model-markov-chain

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Model Markov Chain

Construct, classify, and analyze discrete-time or continuous-time Markov chains from raw transition data or domain specifications, producing stationary distributions, mean first passage times, and simulation-based validation. Covers both DTMC and CTMC workflows end-to-end.

## When to Use

- You need to model a system whose future state depends only on its current state (memoryless property)
- You have observed transition counts or rates between a finite set of states
- You want to compute long-run steady-state probabilities for a process
- You need to determine expected hitting times or absorption probabilities
- You are classifying states as transient, recurrent, or absorbing for structural analysis
- You want to compare alternative Markov models for the same system
- You are building a foundation for more advanced models (hidden Markov models, reinforcement learning MDPs)

## Inputs

### Required

| Input | Type | Description |
|---|---|---|
| `state_space` | list/vector | Exhaustive enumeration of all states in the chain |
| `transition_data` | matrix, data frame, or edge list | Raw transition counts, a probability matrix, or a rate matrix (for CTMC) |
| `chain_type` | string | Either `"discrete"` (DTMC) or `"continuous"` (CTMC) |

### Optional

| Input | Type | Default | Description |
|---|---|---|---|
| `initial_distribution` | vector | uniform | Starting state probabilities |
Read the full skill on GitHub

Details

Skill name
model-markov-chain
Path
i18n/caveman-lite/skills/model-markov-chain/SKILL.md
License
MIT
Repo updated
18 November 2026

FAQ

What does the Model Markov Chain skill do?

Build and analyze discrete or continuous Markov chains including transition matrix construction, state classification, stationary distribution computation, and mean first passage times. Use when modeling a memoryless system with observed transition counts or rates, computing long-run steady-state probabilities, determining expected hitting times or absorption probabilities, classifying states as transient or recurrent, or building a foundation for hidden Markov models or reinforcement learning MDPs.

How do I install the Model Markov Chain skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add pjt222/agent-almanac --skill model-markov-chain. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Model Markov Chain popular?

It lives in pjt222/agent-almanac, which has 45 GitHub stars and 4 forks, and is included in 1 curated skill list.

Related skills

Adapt Architecture

45

pjt222/agent-almanac

Execute structural metamorphosis using strangler fig migration, chrysalis phases, and interface preservation. Covers transformation planning, parallel running, progressive cutover, rollback design, and post-metamorphosis stabilization for system architecture evolution. Use when assess-form has classified the system as READY for transformation, when migrating from monolith to microservices, when replacing a core subsystem while dependents continue operating, or when any architectural change must be gradual rather than big-bang.

Software DevelopmentIn 1 list

Adaptic

45

pjt222/agent-almanac

Master skill composing the 5-step synoptic cycle for panoramic synthesis across multiple domains. Orchestrates meditate, expand-awareness, observe, awareness, integrate-gestalt, and express-insight into a coherent process that produces unified understanding rather than sequential compromise. Use when a problem genuinely spans 3+ domains and the interactions between domains matter more than depth in any one, when sequential analysis feels like compromise rather than integration, or before major architectural decisions affecting multiple stakeholders.

OtherIn 1 list

Add Puzzle Type

45

pjt222/agent-almanac

Scaffold a new puzzle type across all 10+ pipeline integration points in jigsawR. Creates the core puzzle module, wires it into the unified pipeline (generation, positioning, rendering, adjacency), adds ggpuzzle geom/stat layers, updates DESCRIPTION and config.yml, extends the Shiny app, and creates a comprehensive test suite. Use when adding a completely new puzzle type to the package or following the 10-point integration checklist to ensure nothing is missed end-to-end.

Testing & QAIn 1 list

Monitor Model Drift

45

pjt222/agent-almanac

Implement model drift monitoring using Evidently AI, statistical tests (PSI, KS), and custom metrics to detect data drift and concept drift in production ML systems. Set up automated alerting and reporting workflows to catch degradation before it impacts business metrics. Use when production models show unexplained performance degradation, when new data distributions differ from training data, when seasonal shifts affect input features, or when regulatory requirements mandate model monitoring.

AI, LLMs & AgentsIn 1 list

Metal

45

pjt222/agent-almanac

Extract the conceptual essence of a repository as skills, agents, and teams — the project's roles, procedures, and coordination patterns expressed as agentskills.io-standard definitions. Reads an arbitrary codebase and produces generalized definitions that capture WHAT the project does and WHO operates it, without replicating HOW it does it. Use when onboarding to a new codebase and wanting to understand its conceptual architecture, when bootstrapping an agentic system from an existing project, when studying a project's organizational DNA for cross-pollination, or when creating a skill/agent/t

AI, LLMs & AgentsIn 1 list

Orchestrate Ml Pipeline

45

pjt222/agent-almanac

Orchestrate end-to-end machine learning pipelines using Prefect or Airflow with DAG construction, task dependencies, retry logic, scheduling, monitoring, and integration with MLflow, DVC, and feature stores for production ML workflows. Use when automating multi-step ML workflows from data ingestion to deployment, scheduling periodic model retraining, coordinating distributed training tasks, or managing retry logic and failure recovery across pipeline stages.

AI, LLMs & AgentsIn 1 list

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.