Databricks Mlflow Evaluation

MLflow GenAI evaluation and agent improvement on Databricks. Use when running mlflow.genai.evaluate(), building scorers or datasets, aligning judges with MemAlign, optimizing prompts, or monitoring production quality.

407 stars 103 forksAI, LLMs & Agents Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add databricks/databricks-agent-skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluation
Manual install
git clone --depth 1 https://github.com/databricks/databricks-agent-skills.git /tmp/databricks-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/databricks-agent-skills/plugins/databricks/antigravity/skills/databricks-mlflow-evaluation ~/.claude/skills/databricks-mlflow-evaluation

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Databricks MLflow evaluation

Read `databricks-core` for authentication and workspace discovery. For tracing,
trace retrieval, observability, and trace-based debugging, use the separate
`databricks-mlflow-tracing` skill. Verify tracing when the selected evaluation mode or scorers require it; reuse
precomputed outputs or existing traces without rerunning the agent.

The guides below are references, not separately installed skills. When a guide
names an upstream MLflow skill, use the matching guide here or the tracing skill.

| Request | Read this guide |
| --- | --- |
| Define quality criteria or scorers | [Scorer design](references/mlflow-evaluation/references/scorer-design.md) |
| Create a dataset and evaluate an agent | [MLflow evaluation](references/mlflow-evaluation/guide.md) |
| Change agent behavior and verify improvement | [Improve an agent](references/mlflow-agent-improvement/guide.md) |

For MLflow API details, consult the existing `databricks-docs` skill and
https://mlflow.org/docs/latest/llms.txt.

For Databricks judge models, prefer `databricks:/system.ai.<model>` with a
`mlflow[databricks]>=3.16.0`. A 404 commonly indicates an older client that
does not understand the system model URI; a 403 on a legacy endpoint name can
mean legacy serving endpoints are disabled. Verify the installed client and
available `system.ai` model before changing credentials or permissions.

For a new evaluation, establish a baseline with a fixed dataset and scorers,
Read the full skill on GitHub

Details

Skill name
databricks-mlflow-evaluation
Path
plugins/databricks/antigravity/skills/databricks-mlflow-evaluation/SKILL.md
Repo updated
18 November 2026

FAQ

What does the Databricks Mlflow Evaluation skill do?

MLflow GenAI evaluation and agent improvement on Databricks. Use when running mlflow.genai.evaluate(), building scorers or datasets, aligning judges with MemAlign, optimizing prompts, or monitoring production quality.

How do I install the Databricks Mlflow Evaluation skill?

Claude Code plugin: /plugin marketplace add databricks/databricks-agent-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Databricks Mlflow Evaluation popular?

It lives in databricks/databricks-agent-skills, which has 407 GitHub stars and 103 forks.

Related skills

Databricks AI Functions

407

databricks/databricks-agent-skills

Use Databricks built-in AI Functions (ai_classify, ai_decide, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity, ai_parse_document, ai_prep_search, ai_query, ai_forecast) to add AI capabilities directly to SQL and PySpark pipelines without managing model endpoints. Also covers document parsing and building custom RAG pipelines (parse → prep_search → index → query).

AI, LLMs & Agents103

Databricks AI Runtime

407

databricks/databricks-agent-skills

Databricks AI Runtime, the `databricks air` CLI commands for submitting and managing GPU training workloads on Databricks serverless compute. Use for: writing and submitting `databricks air` workload YAML, choosing an accelerator type and Python environment, reading Unity Catalog tables as training data with UCDataset, passing hyperparameters and secrets, configuring retries, checking run status, listing/cancelling runs, streaming a run's logs and watching its progress, debugging failed runs, GPU pools, multi-container workloads, custom Docker image setup, and converting a workload to a Databr

AI, LLMs & Agents103

Databricks Aibi Dashboards

407

databricks/databricks-agent-skills

Create Databricks AI/BI dashboards. Must use when creating, updating, or deploying Lakeview dashboards as Databricks Dashboard have a unique json structure. CRITICAL: You MUST test ALL SQL queries via CLI BEFORE deploying. Follow guidelines strictly.

Data & Analytics103

Databricks Model Serving

407

databricks/databricks-agent-skills

Databricks Model Serving endpoint lifecycle and ops. Use when asked to: CRUD serving endpoints (CLI or MLflow Deployments client); configure traffic routing for A/B / canary deploys and zero-downtime version swaps; retrieve OpenAPI schemas; inspect logs, metrics, or permissions; manage legacy AI Gateway rate limits (not Unity Gateway); discover Foundation Model API endpoints at runtime; integrate endpoints into Databricks Apps; or stream from off-platform clients (Vercel AI SDK v6, standalone Node.js). NOT for: Unity Gateway CRUD and management (databricks-unity-gateway), training, MLflow auto

AI, LLMs & Agents103

Databricks Ml Training

407

databricks/databricks-agent-skills

Train ML models on Databricks. Use for: classification/regression/deep-learning (XGBoost, scikit-learn, LightGBM, PyTorch) with Optuna, @prod/@challenger aliases, batch scoring (spark_udf for plain models, fe.score_batch for feature-store-backed), custom PyFunc, custom ResponsesAgent (LangGraph + UC Function/Vector Search); UC feature tables + FeatureLookup + point-in-time joins + Lakebase online store; declarative Feature Views (create_feature, DeltaTableSource, RollingWindow/SlidingWindow/TumblingWindow, materialize_features, streaming Kafka features). NOT for: endpoint ops (databricks-model

AI, LLMs & Agents103

Databricks Unity Gateway

407

databricks/databricks-agent-skills

Use when querying or calling pay-per-token foundation models, or when creating, updating, listing, or querying Unity Gateway (also called Unity AI Gateway) services, including model provider services, MCP services, system.ai services, and three-part Unity Catalog service names (catalog.schema.service). Also use for managing spend budgets and usage alerts on Unity Gateway, and for migrating from legacy AI Gateway. Also use when a workload with Model Serving endpoint permissions (for example CAN_QUERY) gets PERMISSION_DENIED querying a three-part model name, since serving-endpoint access does no

AI, LLMs & Agents103

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.