Apache Spark
Apache Spark is a distributed engine for batch, SQL and streaming data processing, used from Python through PySpark. Use when a user asks to process big data, run distributed computations, build ETL pipelines, analyse data at scale, read Parquet from S3, or stream from Kafka with PySpark.
Install
npx skills add TerminalSkills/skills --skill apache-sparkgit clone --depth 1 https://github.com/TerminalSkills/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/skills/apache-spark ~/.claude/skills/apache-sparkPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Apache Spark ## Overview Apache Spark is a distributed data processing engine for batch jobs, SQL, structured streaming, machine learning (MLlib) and graph work. PySpark is the Python API. It runs locally for development, on a standalone cluster, YARN or Kubernetes, and on managed services (Databricks, EMR, Dataproc). This skill targets Spark 4.x (4.2.0 released July 2026), which needs Java 17, 21 or 25 and Scala 2.13 for JVM code; PySpark 4.2 needs Python 3.10+. ## Instructions ### Step 1: Install and run locally ```bash python3 -m venv .venv && source .venv/bin/activate pip install pyspark # bundles Spark itself; a JDK (17/21/25) must be on PATH java -version # confirm the JDK first pyspark --master "local[4]" # interactive shell with 4 threads spark-submit --master "local[4]" etl/process.py # run a script ``` On a cluster use `spark-submit --master yarn` or `--master k8s://https://k8s-api.acme.internal:6443`; pass dependencies with `--packages group:artifact:version`. ### Step 2: DataFrame operations
Details
- Skill name
apache-spark- Repository
- TerminalSkills/skills
- Path
skills/apache-spark/SKILL.md- License
- Apache-2.0
- Repo updated
- 8 November 2026
FAQ
What does the Apache Spark skill do?
Apache Spark is a distributed engine for batch, SQL and streaming data processing, used from Python through PySpark. Use when a user asks to process big data, run distributed computations, build ETL pipelines, analyse data at scale, read Parquet from S3, or stream from Kafka with PySpark.
How do I install the Apache Spark skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add TerminalSkills/skills --skill apache-spark. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Apache Spark popular?
It lives in TerminalSkills/skills, which has 213 GitHub stars and 34 forks.
Related skills
3dsmax Rendering
213TerminalSkills/skills
Scripts and configures production rendering in Autodesk 3ds Max with the V-Ray and Corona renderers: output size and files, render elements, denoising, light mix, batch and command-line rendering, and network rendering. Use when a user asks to set up a production render, render several cameras in one batch, render from the command line or on a render farm, add render passes for compositing, or cut render time for archviz and product shots.
3dsmax Scripting
213TerminalSkills/skills
Covers scripting Autodesk 3ds Max, the 3D modeling and rendering application, with MAXScript and Python (pymxs): scene manipulation, object creation, material assignment, camera and light setup, batch operations, and file I/O. Use when tasks involve automating repetitive 3ds Max workflows, batch processing scenes, running scripts headless with 3dsmaxbatch, creating custom tools, or scripting scene setup for archviz, product visualization, or VFX.
3proxy
213TerminalSkills/skills
3proxy is a small open-source proxy server that runs HTTP/HTTPS, SOCKS4/5, SNI and TCP/UDP port-mapping proxies from one config file. Use when a user asks to set up an HTTP or SOCKS5 proxy, add proxy users and passwords, write 3proxy access rules, chain or rotate upstream (parent) proxies, limit bandwidth, connections or monthly traffic per user, run 3proxy in Docker, or fix a 3proxy.cfg that will not start.
Atuin
213TerminalSkills/skills
Atuin replaces your shell history file with a searchable SQLite database that records directory, exit code, duration and host for every command, and can sync it end-to-end encrypted between machines. Use when a user asks to "search shell history", "replace Ctrl+R", "sync history across machines", "find the command I ran in this directory", "import bash/zsh history", "see my most used commands" or "self-host Atuin".
Alembic
213TerminalSkills/skills
Manage database migrations for SQLAlchemy with Alembic. Use when a user asks to version database schemas, autogenerate migration scripts from models, run or roll back migrations, set up async SQLAlchemy migrations, or handle schema changes safely in production.
Authjs
213TerminalSkills/skills
Auth.js (formerly NextAuth.js) is an open-source authentication library that adds OAuth sign-in, magic links, credentials and WebAuthn to Next.js, SvelteKit, Express and other frameworks. Use when a user asks to add login with Google or GitHub, set up NextAuth v5, protect routes with middleware or proxy, add roles to the session, or connect a database adapter such as Drizzle or Prisma. Notes that the project is in maintenance mode under Better Auth.
Browse the most popular Claude skills, all databases skills, or write your own with the skills generator.