Databricks Python SDK

Databricks development guidance including Python SDK, Databricks Connect, CLI, and REST API. Use when working with databricks-sdk, databricks-connect, or Databricks APIs.

407 stars 103 forksSoftware Development Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add databricks/databricks-agent-skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add databricks/databricks-agent-skills --skill databricks-python-sdk
Manual install
git clone --depth 1 https://github.com/databricks/databricks-agent-skills.git /tmp/databricks-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/databricks-agent-skills/plugins/databricks/antigravity/skills/databricks-python-sdk ~/.claude/skills/databricks-python-sdk

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Databricks Development Guide

This skill provides guidance for Databricks SDK, Databricks Connect, CLI, and REST API.

**SDK Documentation:** https://databricks-sdk-py.readthedocs.io/en/latest/
**GitHub Repository:** https://github.com/databricks/databricks-sdk-py

---

## Environment Setup

- Use existing virtual environment at `.venv` or use `uv` to create one
- For Spark operations: `uv pip install databricks-connect`
- For SDK operations: `uv pip install databricks-sdk`
- Databricks CLI version should be 0.278.0 or higher

## Configuration

- Default profile name: `DEFAULT`
- Config file: `~/.databrickscfg`
- Environment variables: `DATABRICKS_HOST`, `DATABRICKS_TOKEN`

---

## Databricks Connect (Spark Operations)

Use `databricks-connect` for running Spark code locally against a Databricks cluster.

```python
from databricks.connect import DatabricksSession

# Auto-detects 'DEFAULT' profile from ~/.databrickscfg
spark = DatabricksSession.builder.getOrCreate()

# With explicit profile
spark = DatabricksSession.builder.profile("MY_PROFILE").getOrCreate()

# Use spark as normal
df = spark.sql("SELECT * FROM catalog.schema.table")
df.show()
```

**IMPORTANT:** Do NOT set `.master("local[*]")` - this will cause issues with Databricks Connect.

---

## Direct REST API Access

For operations not yet in SDK or overly complex via SDK, use direct REST API:
Read the full skill on GitHub

Details

Skill name
databricks-python-sdk
Path
plugins/databricks/antigravity/skills/databricks-python-sdk/SKILL.md
Repo updated
18 November 2026

FAQ

What does the Databricks Python SDK skill do?

Databricks development guidance including Python SDK, Databricks Connect, CLI, and REST API. Use when working with databricks-sdk, databricks-connect, or Databricks APIs.

How do I install the Databricks Python SDK skill?

Claude Code plugin: /plugin marketplace add databricks/databricks-agent-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Databricks Python SDK popular?

It lives in databricks/databricks-agent-skills, which has 407 GitHub stars and 103 forks.

Related skills

Databricks AI Functions

407

databricks/databricks-agent-skills

Use Databricks built-in AI Functions (ai_classify, ai_decide, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity, ai_parse_document, ai_prep_search, ai_query, ai_forecast) to add AI capabilities directly to SQL and PySpark pipelines without managing model endpoints. Also covers document parsing and building custom RAG pipelines (parse → prep_search → index → query).

AI, LLMs & Agents103

Databricks AI Runtime

407

databricks/databricks-agent-skills

Databricks AI Runtime, the `databricks air` CLI commands for submitting and managing GPU training workloads on Databricks serverless compute. Use for: writing and submitting `databricks air` workload YAML, choosing an accelerator type and Python environment, reading Unity Catalog tables as training data with UCDataset, passing hyperparameters and secrets, configuring retries, checking run status, listing/cancelling runs, streaming a run's logs and watching its progress, debugging failed runs, GPU pools, multi-container workloads, custom Docker image setup, and converting a workload to a Databr

AI, LLMs & Agents103

Databricks Aibi Dashboards

407

databricks/databricks-agent-skills

Create Databricks AI/BI dashboards. Must use when creating, updating, or deploying Lakeview dashboards as Databricks Dashboard have a unique json structure. CRITICAL: You MUST test ALL SQL queries via CLI BEFORE deploying. Follow guidelines strictly.

Data & Analytics103

Databricks Sandbox

407

databricks/databricks-agent-skills

Databricks Sandbox: on-demand serverless compute for running untrusted or agent-generated code and background tasks. Create/start/stop/delete sandboxes, run commands (synchronous execute_command_sync in all SDKs; streaming execute_command in the Python SDK only), and manage files through the Databricks SDK (Python, Java, Go, JavaScript). Use for Databricks Sandbox, serverless code-execution environments, or coding-agent runtimes.

Software Development103

Databricks Pipelines

407

databricks/databricks-agent-skills

Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.

Software Development103

Databricks Serverless Migration API

407

databricks/databricks-agent-skills

Migrate Databricks jobs to serverless through the Compute Migrations API and CLI, then track, review, and approve each migration. Use for automated job migrations managed by this service, scripts that start and track migrations, requests to inspect, revise, approve, or cancel migrations, and finding migration backups. For direct code or notebook edits and compatibility analysis, use databricks-serverless-migration.

Software Development103

Browse the most popular Claude skills, all software development skills, or write your own with the skills generator.