Databricks Serverless

Serverless compute for Databricks jobs, Lakeflow pipelines and Declarative Automation Bundles (DABs), with STANDARD or PERFORMANCE_OPTIMIZED and classic only where serverless cannot run the workload. Use when creating, deploying, scheduling or editing a job or pipeline, or deciding its compute. Invoke BEFORE writing a job spec. For migrating existing classic workloads, use databricks-serverless-migration.

407 stars 103 forksDevOps & Cloud Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add databricks/databricks-agent-skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add databricks/databricks-agent-skills --skill databricks-serverless
Manual install
git clone --depth 1 https://github.com/databricks/databricks-agent-skills.git /tmp/databricks-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/databricks-agent-skills/plugins/databricks/antigravity/skills/databricks-serverless ~/.claude/skills/databricks-serverless

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Deploying Databricks jobs and pipelines on serverless

**FIRST**: Use the parent `databricks-core` skill for CLI basics, authentication and profile selection.

This skill decides compute. Where another skill or example (for example `databricks-jobs`) shows
cluster configuration, follow this skill instead.

Serverless is the default compute for every job, pipeline and bundle you create. If the user
explicitly asks for a cluster, do what they ask and mention the serverless option in one line.
When editing an existing job, keep its compute unless the user asks to change it; tasks you add
follow this skill.

## 1. Serverless or classic

Use classic compute only for:

- R code, or Scala in notebook cells (`%scala`): serverless notebooks run Python and SQL only;
- custom Docker images;
- OS packages (`apt-get`) or native libraries with no pip equivalent;
- custom Spark data source JARs;
- a stated hard requirement for an instance type or Databricks Runtime version;
- an explicit user request;
- existing code over the budget in section 5;
- a workspace without serverless: if creating or running the job fails because serverless is not
  available there, put the whole job on classic and say so.

Then give only the blocked task a cluster (one small `job_clusters` entry with the latest LTS
runtime and `autoscale: {min_workers: 1, max_workers: 4}`), keep every other task serverless, and
state the blocker in one line. For a blocked pipeline, give the pipeline a `clusters` block instead
Read the full skill on GitHub

Details

Skill name
databricks-serverless
Path
plugins/databricks/antigravity/skills/databricks-serverless/SKILL.md
Repo updated
18 November 2026

FAQ

What does the Databricks Serverless skill do?

Serverless compute for Databricks jobs, Lakeflow pipelines and Declarative Automation Bundles (DABs), with STANDARD or PERFORMANCE_OPTIMIZED and classic only where serverless cannot run the workload. Use when creating, deploying, scheduling or editing a job or pipeline, or deciding its compute. Invoke BEFORE writing a job spec. For migrating existing classic workloads, use databricks-serverless-migration.

How do I install the Databricks Serverless skill?

Claude Code plugin: /plugin marketplace add databricks/databricks-agent-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Databricks Serverless popular?

It lives in databricks/databricks-agent-skills, which has 407 GitHub stars and 103 forks.

Related skills

Databricks AI Functions

407

databricks/databricks-agent-skills

Use Databricks built-in AI Functions (ai_classify, ai_decide, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity, ai_parse_document, ai_prep_search, ai_query, ai_forecast) to add AI capabilities directly to SQL and PySpark pipelines without managing model endpoints. Also covers document parsing and building custom RAG pipelines (parse → prep_search → index → query).

AI, LLMs & Agents103

Databricks AI Runtime

407

databricks/databricks-agent-skills

Databricks AI Runtime, the `databricks air` CLI commands for submitting and managing GPU training workloads on Databricks serverless compute. Use for: writing and submitting `databricks air` workload YAML, choosing an accelerator type and Python environment, reading Unity Catalog tables as training data with UCDataset, passing hyperparameters and secrets, configuring retries, checking run status, listing/cancelling runs, streaming a run's logs and watching its progress, debugging failed runs, GPU pools, multi-container workloads, custom Docker image setup, and converting a workload to a Databr

AI, LLMs & Agents103

Databricks Aibi Dashboards

407

databricks/databricks-agent-skills

Create Databricks AI/BI dashboards. Must use when creating, updating, or deploying Lakeview dashboards as Databricks Dashboard have a unique json structure. CRITICAL: You MUST test ALL SQL queries via CLI BEFORE deploying. Follow guidelines strictly.

Data & Analytics103

Databricks Serverless Migration

407

databricks/databricks-agent-skills

Make direct code and configuration changes to migrate Databricks workloads from classic compute to serverless. Use for editing notebooks, jobs, pipelines, or Scala JARs (`spark_jar_task`), analyzing serverless compatibility, or writing serverless-compatible code. Provides concrete fixes for the serverless Spark Connect architecture and guides the full migration. For automated job migrations managed through the Compute Migrations API and CLI, use databricks-serverless-migration-api. Not for classic DBR version upgrades or cluster configuration changes within classic compute.

DevOps & Cloud103

Databricks Mlflow Tracing

407

databricks/databricks-agent-skills

MLflow tracing and observability on Databricks. Use when getting started with MLflow, instrumenting GenAI apps and agents, retrieving and analyzing traces, debugging chat sessions, or querying aggregate trace metrics on supported OSS servers.

DevOps & Cloud103

Databricks Setup Local

407

databricks/databricks-agent-skills

Previews, provisions, or diagnoses a uv-managed local Python .venv with `databricks environments setup-local`. Use when the user wants to set up or fix one for Databricks Connect, cluster or serverless compute, `--job-task`, or a bundle target, or when setup-local fails.

DevOps & Cloud103

Browse the most popular Claude skills, all devops & cloud skills, or write your own with the skills generator.