Data Engineer

Build scalable data pipelines, modern data warehouses, and real-time streaming architectures. Implements Apache Spark, dbt, Airflow, and cloud-native data platforms. Use for data pipeline design, analytics infrastructure, or modern data stack implementation.

2,047 stars 295 forksData & Analytics

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add rmyndharis/antigravity-skills --skill data-engineer
Manual install
git clone --depth 1 https://github.com/rmyndharis/antigravity-skills.git /tmp/antigravity-skills
mkdir -p ~/.claude/skills
cp -r /tmp/antigravity-skills/skills/data-engineer ~/.claude/skills/data-engineer

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

You are a data engineer specializing in scalable data pipelines, modern data architecture, and analytics infrastructure.

## Use this skill when

- Designing batch or streaming data pipelines
- Building data warehouses or lakehouse architectures
- Implementing data quality, lineage, or governance

## Do not use this skill when

- You only need exploratory data analysis
- You are doing ML model development without pipelines
- You cannot access data sources or storage systems

## Instructions

1. Define sources, SLAs, and data contracts.
2. Choose architecture, storage, and orchestration tools.
3. Implement ingestion, transformation, and validation.
4. Monitor quality, costs, and operational reliability.

## Safety

- Protect PII and enforce least-privilege access.
- Validate data before writing to production sinks.

## Purpose
Expert data engineer specializing in building robust, scalable data pipelines and modern data platforms. Masters the complete modern data stack including batch and streaming processing, data warehousing, lakehouse architectures, and cloud-native data services. Focuses on reliable, performant, and cost-effective data solutions.

## Capabilities

### Modern Data Stack & Architecture
- Data lakehouse architectures with Delta Lake, Apache Iceberg, and Apache Hudi
- Cloud data warehouses: Snowflake, BigQuery, Redshift, Databricks SQL
- Data lakes: AWS S3, Azure Data Lake, Google Cloud Storage with structured organization
Read the full skill on GitHub

Details

Skill name
data-engineer
Path
skills/data-engineer/SKILL.md
License
MIT
Repo updated
7 November 2026
Copies
Found in 2 GitHub repos

FAQ

What does the Data Engineer skill do?

Build scalable data pipelines, modern data warehouses, and real-time streaming architectures. Implements Apache Spark, dbt, Airflow, and cloud-native data platforms. Use for data pipeline design, analytics infrastructure, or modern data stack implementation.

How do I install the Data Engineer skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add rmyndharis/antigravity-skills --skill data-engineer. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Data Engineer popular?

It lives in rmyndharis/antigravity-skills, which has 2,047 GitHub stars and 295 forks.

Related skills

Airflow Dag Patterns

2k

rmyndharis/antigravity-skills

Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.

Testing & QA295

Agent Orchestration Improve Agent

2k

rmyndharis/antigravity-skills

Systematic improvement of existing agents through performance analysis, prompt engineering, and continuous iteration.

AI, LLMs & Agents295

Agent Orchestration Multi Agent Optimize

2k

rmyndharis/antigravity-skills

Optimize multi-agent systems with coordinated profiling, workload distribution, and cost-aware orchestration. Use when improving agent performance, throughput, or reliability.

AI, LLMs & Agents295

Data Engineering Data Driven Feature

2k

rmyndharis/antigravity-skills

Build features guided by data insights, A/B testing, and continuous measurement using specialized agents for analysis, implementation, and experimentation.

Data & Analytics295

Crw Watch

1.3k

fastcrw/crw

Detect what changed between two page snapshots with fastCRW — stateless diff as a REST primitive. Use when you need to track content changes, monitor a page for updates, or build a cron-based alert system: "has this page changed?", "alert me when pricing changes", "diff this week's scrape against last week's". Step 7 of the crw workflow ladder.

Data & AnalyticsIn 1 list102

Data Scientist

2k

rmyndharis/antigravity-skills

Expert data scientist for advanced analytics, machine learning, and statistical modeling. Handles complex data analysis, predictive modeling, and business intelligence. Use for data analysis tasks, ML modeling, statistical analysis, and data-driven insights.

Data & Analytics295

Browse the most popular Claude skills, all data & analytics skills, or write your own with the skills generator.