Databricks Pipelines

Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.

407 stars 103 forksSoftware Development Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add databricks/databricks-agent-skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add databricks/databricks-agent-skills --skill databricks-pipelines
Manual install
git clone --depth 1 https://github.com/databricks/databricks-agent-skills.git /tmp/databricks-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/databricks-agent-skills/plugins/databricks/antigravity/skills/databricks-pipelines ~/.claude/skills/databricks-pipelines

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Lakeflow Spark Declarative Pipelines Development

**FIRST**: Use the parent `databricks-core` skill for CLI basics, authentication, profile selection, and data discovery commands.

SDP, LDP, Lakeflow Declarative Pipelines, and DLT (former name) all refer to the same product.

## Decision Tree

Use this tree to choose the dataset type first, then layer on applicable features. Multiple features can apply to the same dataset — e.g., a Streaming Table can use Auto Loader for ingestion, Append Flows for fan-in, and Expectations for data quality.
Read the full skill on GitHub

Details

Skill name
databricks-pipelines
Path
plugins/databricks/antigravity/skills/databricks-pipelines/SKILL.md
Repo updated
18 November 2026

FAQ

What does the Databricks Pipelines skill do?

Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.

How do I install the Databricks Pipelines skill?

Claude Code plugin: /plugin marketplace add databricks/databricks-agent-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Databricks Pipelines popular?

It lives in databricks/databricks-agent-skills, which has 407 GitHub stars and 103 forks.

Related skills

Databricks AI Functions

407

databricks/databricks-agent-skills

Use Databricks built-in AI Functions (ai_classify, ai_decide, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity, ai_parse_document, ai_prep_search, ai_query, ai_forecast) to add AI capabilities directly to SQL and PySpark pipelines without managing model endpoints. Also covers document parsing and building custom RAG pipelines (parse → prep_search → index → query).

AI, LLMs & Agents103

Databricks AI Runtime

407

databricks/databricks-agent-skills

Databricks AI Runtime, the `databricks air` CLI commands for submitting and managing GPU training workloads on Databricks serverless compute. Use for: writing and submitting `databricks air` workload YAML, choosing an accelerator type and Python environment, reading Unity Catalog tables as training data with UCDataset, passing hyperparameters and secrets, configuring retries, checking run status, listing/cancelling runs, streaming a run's logs and watching its progress, debugging failed runs, GPU pools, multi-container workloads, custom Docker image setup, and converting a workload to a Databr

AI, LLMs & Agents103

Databricks Aibi Dashboards

407

databricks/databricks-agent-skills

Create Databricks AI/BI dashboards. Must use when creating, updating, or deploying Lakeview dashboards as Databricks Dashboard have a unique json structure. CRITICAL: You MUST test ALL SQL queries via CLI BEFORE deploying. Follow guidelines strictly.

Data & Analytics103

Databricks Python SDK

407

databricks/databricks-agent-skills

Databricks development guidance including Python SDK, Databricks Connect, CLI, and REST API. Use when working with databricks-sdk, databricks-connect, or Databricks APIs.

Software Development103

Databricks Core

407

databricks/databricks-agent-skills

Databricks CLI operations and the parent/entry-point skill for Databricks CLI use: authentication, profile selection, and bundles. Load this first for CLI, auth, profile, and bundle tasks, then load the matching product skill. For finding or exploring data, answering questions about the data, or generating SQL, load the databricks-data-discovery skill (it routes to Genie One). Contains up-to-date guidelines for Databricks-related CLI tasks.

Software Development103

Databricks Sandbox

407

databricks/databricks-agent-skills

Databricks Sandbox: on-demand serverless compute for running untrusted or agent-generated code and background tasks. Create/start/stop/delete sandboxes, run commands (synchronous execute_command_sync in all SDKs; streaming execute_command in the Python SDK only), and manage files through the Databricks SDK (Python, Java, Go, JavaScript). Use for Databricks Sandbox, serverless code-execution environments, or coding-agent runtimes.

Software Development103

Browse the most popular Claude skills, all software development skills, or write your own with the skills generator.