Senior Data Engineer
Data engineering for batch and streaming pipelines with Airflow, dbt, Spark, and Kafka. Use when designing data architectures, building pipelines, adding data-quality checks, optimizing ETL/ELT, or troubleshooting pipeline failures.
Install
/plugin marketplace add borghei/Claude-SkillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add borghei/Claude-Skills --skill senior-data-engineergit clone --depth 1 https://github.com/borghei/Claude-Skills.git /tmp/Claude-Skills
mkdir -p ~/.claude/skills
cp -r /tmp/Claude-Skills/engineering/senior-data-engineer ~/.claude/skills/senior-data-engineerPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Senior Data Engineer Generate pipeline configurations (Airflow, Prefect, Dagster), validate data quality with profiling and anomaly detection, and optimize SQL/Spark performance with actionable recommendations. ## Core Capabilities - **Pipeline generation** — Airflow/Prefect/Dagster DAG code for batch and incremental loads, with DAG validation. - **Data quality** — schema validation, profiling, anomaly detection, data contracts, and Great Expectations suite generation. - **ETL/ELT optimization** — SQL and Spark analysis, partition strategy, and query cost estimation per warehouse. - **Architecture decisions** — batch vs streaming and warehouse vs lakehouse trade-off frameworks. - **Reliability patterns** — incremental watermarks, dead letter queues, freshness checks, and schema-drift detection. ## When to Use - Designing a data architecture or choosing batch vs streaming / warehouse vs lakehouse. - Building or generating Airflow/Spark/dbt pipelines. - Adding data-quality checks or data contracts. - Optimizing slow ETL/ELT queries or troubleshooting pipeline failures. ## Clarify First Before generating pipelines, settle the inputs below. When one is missing or vague and would change the result, ask about it; when it wouldn't, use a sensible default and say which one you used. - **Orchestrator:** Airflow / Prefect / Dagster (`--type`; changes the generated DAG code).
Details
- Skill name
senior-data-engineer- Repository
- borghei/Claude-Skills
- Path
engineering/senior-data-engineer/SKILL.md- Repo updated
- 8 November 2026
- Listed in
- BehiSecc/awesome-claude-skills
FAQ
What does the Senior Data Engineer skill do?
Data engineering for batch and streaming pipelines with Airflow, dbt, Spark, and Kafka. Use when designing data architectures, building pipelines, adding data-quality checks, optimizing ETL/ELT, or troubleshooting pipeline failures.
How do I install the Senior Data Engineer skill?
Claude Code plugin: /plugin marketplace add borghei/Claude-Skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Senior Data Engineer popular?
It lives in borghei/Claude-Skills, which has 1,157 GitHub stars and 217 forks, and is included in 1 curated skill list.
Related skills
Cto Advisor
1.2kborghei/Claude-Skills
Technical leadership guidance for engineering teams, architecture decisions, and technology strategy. Use when assessing technical debt, scaling engineering teams, evaluating technologies, or making architecture decisions.
Customer Success Manager
1.2kborghei/Claude-Skills
Monitors customer health, predicts churn risk, and identifies expansion opportunities using weighted scoring models for SaaS customer success
Financial Analyst
1.2kborghei/Claude-Skills
Performs financial ratio analysis, DCF valuation, budget variance analysis, and rolling forecast construction for strategic decision-making
Statistical Analyst
1.2kborghei/Claude-Skills
Applied statistics for business and product questions — test selection, assumption checks, power planning, effect sizes with intervals, multiplicity correction. Use when interpreting an experiment, sizing a study, or vetting a claim.
Product Analytics
1.2kborghei/Claude-Skills
Product analytics for instrumenting products, defining metrics, and building retention funnels. Use when designing a metric tree, instrumenting a feature, auditing instrumentation, defining a North Star, or building an analytics roadmap.
Wayfinder
1.2kborghei/Claude-Skills
Chart work too big for one sprint as a map of decision tickets, resolve them in dependency order, then turn answers into tracer-bullet tickets. Use when the route is unclear, open questions block each other, or estimates swing wildly.
Browse the most popular Claude skills, all data & analytics skills, or write your own with the skills generator.