Databricks Mlflow Evaluation
MLflow GenAI evaluation and agent improvement on Databricks. Use when running mlflow.genai.evaluate(), building scorers or datasets, aligning judges with MemAlign, optimizing prompts, or monitoring production quality.
Install
/plugin marketplace add databricks/databricks-agent-skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add databricks/databricks-agent-skills --skill databricks-mlflow-evaluationgit clone --depth 1 https://github.com/databricks/databricks-agent-skills.git /tmp/databricks-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/databricks-agent-skills/plugins/databricks/antigravity/skills/databricks-mlflow-evaluation ~/.claude/skills/databricks-mlflow-evaluationPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Databricks MLflow evaluation Read `databricks-core` for authentication and workspace discovery. For tracing, trace retrieval, observability, and trace-based debugging, use the separate `databricks-mlflow-tracing` skill. Verify tracing when the selected evaluation mode or scorers require it; reuse precomputed outputs or existing traces without rerunning the agent. The guides below are references, not separately installed skills. When a guide names an upstream MLflow skill, use the matching guide here or the tracing skill. | Request | Read this guide | | --- | --- | | Define quality criteria or scorers | [Scorer design](references/mlflow-evaluation/references/scorer-design.md) | | Create a dataset and evaluate an agent | [MLflow evaluation](references/mlflow-evaluation/guide.md) | | Change agent behavior and verify improvement | [Improve an agent](references/mlflow-agent-improvement/guide.md) | For MLflow API details, consult the existing `databricks-docs` skill and https://mlflow.org/docs/latest/llms.txt. For Databricks judge models, prefer `databricks:/system.ai.<model>` with a `mlflow[databricks]>=3.16.0`. A 404 commonly indicates an older client that does not understand the system model URI; a 403 on a legacy endpoint name can mean legacy serving endpoints are disabled. Verify the installed client and available `system.ai` model before changing credentials or permissions. For a new evaluation, establish a baseline with a fixed dataset and scorers,
Details
- Skill name
databricks-mlflow-evaluation- Repository
- databricks/databricks-agent-skills
- Path
plugins/databricks/antigravity/skills/databricks-mlflow-evaluation/SKILL.md- Repo updated
- 18 November 2026
FAQ
What does the Databricks Mlflow Evaluation skill do?
MLflow GenAI evaluation and agent improvement on Databricks. Use when running mlflow.genai.evaluate(), building scorers or datasets, aligning judges with MemAlign, optimizing prompts, or monitoring production quality.
How do I install the Databricks Mlflow Evaluation skill?
Claude Code plugin: /plugin marketplace add databricks/databricks-agent-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Databricks Mlflow Evaluation popular?
It lives in databricks/databricks-agent-skills, which has 407 GitHub stars and 103 forks.
Related skills
Databricks AI Functions
407databricks/databricks-agent-skills
Use Databricks built-in AI Functions (ai_classify, ai_decide, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity, ai_parse_document, ai_prep_search, ai_query, ai_forecast) to add AI capabilities directly to SQL and PySpark pipelines without managing model endpoints. Also covers document parsing and building custom RAG pipelines (parse → prep_search → index → query).
Databricks AI Runtime
407databricks/databricks-agent-skills
Databricks AI Runtime, the `databricks air` CLI commands for submitting and managing GPU training workloads on Databricks serverless compute. Use for: writing and submitting `databricks air` workload YAML, choosing an accelerator type and Python environment, reading Unity Catalog tables as training data with UCDataset, passing hyperparameters and secrets, configuring retries, checking run status, listing/cancelling runs, streaming a run's logs and watching its progress, debugging failed runs, GPU pools, multi-container workloads, custom Docker image setup, and converting a workload to a Databr
Databricks Aibi Dashboards
407databricks/databricks-agent-skills
Create Databricks AI/BI dashboards. Must use when creating, updating, or deploying Lakeview dashboards as Databricks Dashboard have a unique json structure. CRITICAL: You MUST test ALL SQL queries via CLI BEFORE deploying. Follow guidelines strictly.
Databricks Model Serving
407databricks/databricks-agent-skills
Databricks Model Serving endpoint lifecycle and ops. Use when asked to: CRUD serving endpoints (CLI or MLflow Deployments client); configure traffic routing for A/B / canary deploys and zero-downtime version swaps; retrieve OpenAPI schemas; inspect logs, metrics, or permissions; manage legacy AI Gateway rate limits (not Unity Gateway); discover Foundation Model API endpoints at runtime; integrate endpoints into Databricks Apps; or stream from off-platform clients (Vercel AI SDK v6, standalone Node.js). NOT for: Unity Gateway CRUD and management (databricks-unity-gateway), training, MLflow auto
Databricks Ml Training
407databricks/databricks-agent-skills
Train ML models on Databricks. Use for: classification/regression/deep-learning (XGBoost, scikit-learn, LightGBM, PyTorch) with Optuna, @prod/@challenger aliases, batch scoring (spark_udf for plain models, fe.score_batch for feature-store-backed), custom PyFunc, custom ResponsesAgent (LangGraph + UC Function/Vector Search); UC feature tables + FeatureLookup + point-in-time joins + Lakebase online store; declarative Feature Views (create_feature, DeltaTableSource, RollingWindow/SlidingWindow/TumblingWindow, materialize_features, streaming Kafka features). NOT for: endpoint ops (databricks-model
Databricks Unity Gateway
407databricks/databricks-agent-skills
Use when querying or calling pay-per-token foundation models, or when creating, updating, listing, or querying Unity Gateway (also called Unity AI Gateway) services, including model provider services, MCP services, system.ai services, and three-part Unity Catalog service names (catalog.schema.service). Also use for managing spend budgets and usage alerts on Unity Gateway, and for migrating from legacy AI Gateway. Also use when a workload with Model Serving endpoint permissions (for example CAN_QUERY) gets PERMISSION_DENIED querying a three-part model name, since serving-endpoint access does no
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.