Databricks Python SDK
Databricks development guidance including Python SDK, Databricks Connect, CLI, and REST API. Use when working with databricks-sdk, databricks-connect, or Databricks APIs.
Install
/plugin marketplace add databricks/databricks-agent-skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add databricks/databricks-agent-skills --skill databricks-python-sdkgit clone --depth 1 https://github.com/databricks/databricks-agent-skills.git /tmp/databricks-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/databricks-agent-skills/plugins/databricks/antigravity/skills/databricks-python-sdk ~/.claude/skills/databricks-python-sdkPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Databricks Development Guide
This skill provides guidance for Databricks SDK, Databricks Connect, CLI, and REST API.
**SDK Documentation:** https://databricks-sdk-py.readthedocs.io/en/latest/
**GitHub Repository:** https://github.com/databricks/databricks-sdk-py
---
## Environment Setup
- Use existing virtual environment at `.venv` or use `uv` to create one
- For Spark operations: `uv pip install databricks-connect`
- For SDK operations: `uv pip install databricks-sdk`
- Databricks CLI version should be 0.278.0 or higher
## Configuration
- Default profile name: `DEFAULT`
- Config file: `~/.databrickscfg`
- Environment variables: `DATABRICKS_HOST`, `DATABRICKS_TOKEN`
---
## Databricks Connect (Spark Operations)
Use `databricks-connect` for running Spark code locally against a Databricks cluster.
```python
from databricks.connect import DatabricksSession
# Auto-detects 'DEFAULT' profile from ~/.databrickscfg
spark = DatabricksSession.builder.getOrCreate()
# With explicit profile
spark = DatabricksSession.builder.profile("MY_PROFILE").getOrCreate()
# Use spark as normal
df = spark.sql("SELECT * FROM catalog.schema.table")
df.show()
```
**IMPORTANT:** Do NOT set `.master("local[*]")` - this will cause issues with Databricks Connect.
---
## Direct REST API Access
For operations not yet in SDK or overly complex via SDK, use direct REST API:Details
- Skill name
databricks-python-sdk- Repository
- databricks/databricks-agent-skills
- Path
plugins/databricks/antigravity/skills/databricks-python-sdk/SKILL.md- Repo updated
- 18 November 2026
FAQ
What does the Databricks Python SDK skill do?
Databricks development guidance including Python SDK, Databricks Connect, CLI, and REST API. Use when working with databricks-sdk, databricks-connect, or Databricks APIs.
How do I install the Databricks Python SDK skill?
Claude Code plugin: /plugin marketplace add databricks/databricks-agent-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Databricks Python SDK popular?
It lives in databricks/databricks-agent-skills, which has 407 GitHub stars and 103 forks.
Related skills
Databricks AI Functions
407databricks/databricks-agent-skills
Use Databricks built-in AI Functions (ai_classify, ai_decide, ai_extract, ai_summarize, ai_mask, ai_translate, ai_fix_grammar, ai_gen, ai_analyze_sentiment, ai_similarity, ai_parse_document, ai_prep_search, ai_query, ai_forecast) to add AI capabilities directly to SQL and PySpark pipelines without managing model endpoints. Also covers document parsing and building custom RAG pipelines (parse → prep_search → index → query).
Databricks AI Runtime
407databricks/databricks-agent-skills
Databricks AI Runtime, the `databricks air` CLI commands for submitting and managing GPU training workloads on Databricks serverless compute. Use for: writing and submitting `databricks air` workload YAML, choosing an accelerator type and Python environment, reading Unity Catalog tables as training data with UCDataset, passing hyperparameters and secrets, configuring retries, checking run status, listing/cancelling runs, streaming a run's logs and watching its progress, debugging failed runs, GPU pools, multi-container workloads, custom Docker image setup, and converting a workload to a Databr
Databricks Aibi Dashboards
407databricks/databricks-agent-skills
Create Databricks AI/BI dashboards. Must use when creating, updating, or deploying Lakeview dashboards as Databricks Dashboard have a unique json structure. CRITICAL: You MUST test ALL SQL queries via CLI BEFORE deploying. Follow guidelines strictly.
Databricks Sandbox
407databricks/databricks-agent-skills
Databricks Sandbox: on-demand serverless compute for running untrusted or agent-generated code and background tasks. Create/start/stop/delete sandboxes, run commands (synchronous execute_command_sync in all SDKs; streaming execute_command in the Python SDK only), and manage files through the Databricks SDK (Python, Java, Go, JavaScript). Use for Databricks Sandbox, serverless code-execution environments, or coding-agent runtimes.
Databricks Pipelines
407databricks/databricks-agent-skills
Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL, including Auto CDC from event streams or periodic complete snapshots. Invoke BEFORE starting implementation.
Databricks Serverless Migration API
407databricks/databricks-agent-skills
Migrate Databricks jobs to serverless through the Compute Migrations API and CLI, then track, review, and approve each migration. Use for automated job migrations managed by this service, scripts that start and track migrations, requests to inspect, revise, approve, or cancel migrations, and finding migration backups. For direct code or notebook edits and compatibility analysis, use databricks-serverless-migration.
Browse the most popular Claude skills, all software development skills, or write your own with the skills generator.