Databricks Pipelines
Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.
Install
/plugin marketplace add databricks/databricks-agent-skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add databricks/databricks-agent-skills --skill databricks-pipelinesgit clone --depth 1 https://github.com/databricks/databricks-agent-skills.git /tmp/databricks-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/databricks-agent-skills/plugins/databricks/antigravity/skills/databricks-pipelines ~/.claude/skills/databricks-pipelinesPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Lakeflow Spark Declarative Pipelines Development **FIRST**: Use the parent `databricks-core` skill for CLI basics, authentication, profile selection, and data discovery commands. SDP, LDP, Lakeflow Declarative Pipelines, and DLT (former name) all refer to the same product. ## Decision Tree Use this tree to choose the dataset type first, then layer on applicable features. Multiple features can apply to the same dataset — e.g., a Streaming Table can use Auto Loader for ingestion, Append Flows for fan-in, and Expectations for data quality.
Details
- Skill name
databricks-pipelines- Repository
- databricks/databricks-agent-skills
- Path
plugins/databricks/antigravity/skills/databricks-pipelines/SKILL.md- Repo updated
- 18 November 2026
FAQ
What does the Databricks Pipelines skill do?
Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.
How do I install the Databricks Pipelines skill?
Claude Code plugin: /plugin marketplace add databricks/databricks-agent-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Databricks Pipelines popular?
It lives in databricks/databricks-agent-skills, which has 407 GitHub stars and 103 forks.
Related skills
Databricks AI Functions
407databricks/databricks-agent-skills
Use Databricks built-in AI Functions (ai_classify, ai_decide, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity, ai_parse_document, ai_prep_search, ai_query, ai_forecast) to add AI capabilities directly to SQL and PySpark pipelines without managing model endpoints. Also covers document parsing and building custom RAG pipelines (parse → prep_search → index → query).
Databricks AI Runtime
407databricks/databricks-agent-skills
Databricks AI Runtime, the `databricks air` CLI commands for submitting and managing GPU training workloads on Databricks serverless compute. Use for: writing and submitting `databricks air` workload YAML, choosing an accelerator type and Python environment, reading Unity Catalog tables as training data with UCDataset, passing hyperparameters and secrets, configuring retries, checking run status, listing/cancelling runs, streaming a run's logs and watching its progress, debugging failed runs, GPU pools, multi-container workloads, custom Docker image setup, and converting a workload to a Databr
Databricks Aibi Dashboards
407databricks/databricks-agent-skills
Create Databricks AI/BI dashboards. Must use when creating, updating, or deploying Lakeview dashboards as Databricks Dashboard have a unique json structure. CRITICAL: You MUST test ALL SQL queries via CLI BEFORE deploying. Follow guidelines strictly.
Databricks Python SDK
407databricks/databricks-agent-skills
Databricks development guidance including Python SDK, Databricks Connect, CLI, and REST API. Use when working with databricks-sdk, databricks-connect, or Databricks APIs.
Databricks Core
407databricks/databricks-agent-skills
Databricks CLI operations and the parent/entry-point skill for Databricks CLI use: authentication, profile selection, and bundles. Load this first for CLI, auth, profile, and bundle tasks, then load the matching product skill. For finding or exploring data, answering questions about the data, or generating SQL, load the databricks-data-discovery skill (it routes to Genie One). Contains up-to-date guidelines for Databricks-related CLI tasks.
Databricks Sandbox
407databricks/databricks-agent-skills
Databricks Sandbox: on-demand serverless compute for running untrusted or agent-generated code and background tasks. Create/start/stop/delete sandboxes, run commands (synchronous execute_command_sync in all SDKs; streaming execute_command in the Python SDK only), and manage files through the Databricks SDK (Python, Java, Go, JavaScript). Use for Databricks Sandbox, serverless code-execution environments, or coding-agent runtimes.
Browse the most popular Claude skills, all software development skills, or write your own with the skills generator.