Track Ml Experiments

Set up MLflow tracking server for experiment management, configure autologging for popular ML frameworks, compare runs with metrics and visualizations, and manage artifacts in remote storage backends for reproducible machine learning workflows. Use when starting a new ML project that requires experiment tracking, migrating from manual logs to automated tracking, comparing multiple training runs systematically, or building reproducible ML workflows with full lineage tracking.

45 stars 4 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add pjt222/agent-almanac --skill track-ml-experiments
Manual install
git clone --depth 1 https://github.com/pjt222/agent-almanac.git /tmp/agent-almanac
mkdir -p ~/.claude/skills
cp -r /tmp/agent-almanac/i18n/caveman-lite/skills/track-ml-experiments ~/.claude/skills/track-ml-experiments

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Track ML Experiments

> See [Extended Examples](references/EXAMPLES.md) for complete configuration files and templates.

Set up MLflow tracking server and implement comprehensive experiment tracking with metrics, parameters, and artifacts.

## When to Use

- Starting a new machine learning project requiring experiment tracking
- Migrating from manual experiment logs to automated tracking
- Comparing multiple model training runs systematically
- Sharing experiment results with team members
- Building reproducible ML workflows with full lineage tracking
- Integrating experiment tracking into CI/CD pipelines

## Inputs

- **Required**: Python environment with ML framework (sklearn, pytorch, tensorflow, xgboost)
- **Required**: MLflow installation (`pip install mlflow`)
- **Optional**: Remote storage backend (S3, Azure Blob, GCS) for artifacts
- **Optional**: Database backend (PostgreSQL, MySQL) for metadata storage
- **Optional**: Authentication credentials for remote backends

## Procedure

### Step 1: Initialize MLflow Tracking Server

Set up the MLflow tracking server with appropriate backend stores.

```bash
# Option 1: Local file-based tracking (development)
mkdir -p mlruns
export MLFLOW_TRACKING_URI="file:./mlruns"

# Option 2: SQLite backend with local artifacts
mlflow server \
  --backend-store-uri sqlite:///mlflow.db \
  --default-artifact-root ./mlartifacts \
# ... (see EXAMPLES.md for complete implementation)
```

Create a configuration file for team sharing:
Read the full skill on GitHub

Details

Skill name
track-ml-experiments
Path
i18n/caveman-lite/skills/track-ml-experiments/SKILL.md
License
MIT
Repo updated
18 November 2026

FAQ

What does the Track Ml Experiments skill do?

Set up MLflow tracking server for experiment management, configure autologging for popular ML frameworks, compare runs with metrics and visualizations, and manage artifacts in remote storage backends for reproducible machine learning workflows. Use when starting a new ML project that requires experiment tracking, migrating from manual logs to automated tracking, comparing multiple training runs systematically, or building reproducible ML workflows with full lineage tracking.

How do I install the Track Ml Experiments skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add pjt222/agent-almanac --skill track-ml-experiments. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Track Ml Experiments popular?

It lives in pjt222/agent-almanac, which has 45 GitHub stars and 4 forks, and is included in 1 curated skill list.

Related skills

Adapt Architecture

45

pjt222/agent-almanac

Execute structural metamorphosis using strangler fig migration, chrysalis phases, and interface preservation. Covers transformation planning, parallel running, progressive cutover, rollback design, and post-metamorphosis stabilization for system architecture evolution. Use when assess-form has classified the system as READY for transformation, when migrating from monolith to microservices, when replacing a core subsystem while dependents continue operating, or when any architectural change must be gradual rather than big-bang.

Software DevelopmentIn 1 list

Adaptic

45

pjt222/agent-almanac

Master skill composing the 5-step synoptic cycle for panoramic synthesis across multiple domains. Orchestrates meditate, expand-awareness, observe, awareness, integrate-gestalt, and express-insight into a coherent process that produces unified understanding rather than sequential compromise. Use when a problem genuinely spans 3+ domains and the interactions between domains matter more than depth in any one, when sequential analysis feels like compromise rather than integration, or before major architectural decisions affecting multiple stakeholders.

OtherIn 1 list

Add Puzzle Type

45

pjt222/agent-almanac

Scaffold a new puzzle type across all 10+ pipeline integration points in jigsawR. Creates the core puzzle module, wires it into the unified pipeline (generation, positioning, rendering, adjacency), adds ggpuzzle geom/stat layers, updates DESCRIPTION and config.yml, extends the Shiny app, and creates a comprehensive test suite. Use when adding a completely new puzzle type to the package or following the 10-point integration checklist to ensure nothing is missed end-to-end.

Testing & QAIn 1 list

Troubleshoot MCP Connection

45

pjt222/agent-almanac

Diagnose and fix MCP server connection issues between Claude Code, Claude Desktop, and MCP servers. Covers Windows argument parsing, authentication failures, transport issues, and platform-specific debugging. Use when Claude Code or Claude Desktop fails to connect to an MCP server, when MCP tools don't appear in sessions, on "cannot attach the server" errors, when a working connection has stopped, or when setting up MCP on a new machine.

AI, LLMs & AgentsIn 1 list

Setup Automl Pipeline

45

pjt222/agent-almanac

Configure automated ML pipelines using Optuna or Ray Tune for hyperparameter optimization. Implement efficient search strategies (Hyperband, ASHA), define search spaces, and set up early stopping to find optimal model configurations with minimal manual tuning. Use when starting a new ML project, retraining with new data and re-optimizing hyperparameters, comparing multiple algorithms, or when the team lacks deep expertise in specific algorithm hyperparameters.

AI, LLMs & AgentsIn 1 list

Unleash the Agents

45

pjt222/agent-almanac

Launch all available agents in parallel waves for open-ended hypothesis generation on problems where the correct domain is unknown. Use when facing a cross-domain problem with no clear starting point, when single-agent approaches have stalled, or when diverse perspectives are more valuable than deep expertise. Produces a ranked hypothesis set with convergence analysis and adversarial refinement.

AI, LLMs & AgentsIn 1 list

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.