Data Engineering Data Pipeline
You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.
Install
npx skills add Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI --skill data-engineering-data-pipelinegit clone --depth 1 https://github.com/Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI.git /tmp/Agent-Skills-4-Vibe-Coding-CLI
mkdir -p ~/.claude/skills
cp -r /tmp/Agent-Skills-4-Vibe-Coding-CLI/.agent/skills/data-engineering-data-pipeline ~/.claude/skills/data-engineering-data-pipelinePersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Data Pipeline Architecture You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing. ## Use this skill when - Working on data pipeline architecture tasks or workflows - Needing guidance, best practices, or checklists for data pipeline architecture ## Do not use this skill when - The task is unrelated to data pipeline architecture - You need a different domain or tool outside this scope ## Requirements $ARGUMENTS ## Core Capabilities - Design ETL/ELT, Lambda, Kappa, and Lakehouse architectures - Implement batch and streaming data ingestion - Build workflow orchestration with Airflow/Prefect - Transform data using dbt and Spark - Manage Delta Lake/Iceberg storage with ACID transactions - Implement data quality frameworks (Great Expectations, dbt tests) - Monitor pipelines with CloudWatch/Prometheus/Grafana - Optimize costs through partitioning, lifecycle policies, and compute optimization ## Instructions ## 🧠 Knowledge Modules (Fractal Skills) ### 1. [1. Architecture Design](./sub-skills/1-architecture-design.md) ### 2. [2. Ingestion Implementation](./sub-skills/2-ingestion-implementation.md) ### 3. [3. Orchestration](./sub-skills/3-orchestration.md) ### 4. [4. Transformation with dbt](./sub-skills/4-transformation-with-dbt.md) ### 5. [5. Data Quality Framework](./sub-skills/5-data-quality-framework.md) ### 6. [6. Storage Strategy](./sub-skills/6-storage-strategy.md)
Details
- Skill name
data-engineering-data-pipeline- Repository
- Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI
- Path
.agent/skills/data-engineering-data-pipeline/SKILL.md- Repo updated
- 11 June 2026
FAQ
What does the Data Engineering Data Pipeline skill do?
You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.
How do I install the Data Engineering Data Pipeline skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI --skill data-engineering-data-pipeline. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Data Engineering Data Pipeline popular?
It lives in Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI, which has 514 GitHub stars and 146 forks.
Related skills
Address Github Comments
514Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI
Use when you need to address review or issue comments on an open GitHub Pull Request using the gh CLI.
Agent Memory MCP
514Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI
A hybrid memory system that provides persistent, searchable knowledge management for AI agents (Architecture, Patterns, Decisions).
API Documentation Generator
514Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI
Generate comprehensive, developer-friendly API documentation from code, including endpoints, parameters, examples, and best practices
Firecrawl Scraper
514Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI
Deep web scraping, screenshots, PDF parsing, and website crawling using Firecrawl API
Analytics Tracking
514Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI
Design, audit, and improve analytics tracking systems that produce reliable, decision-ready data. Use when the user wants to set up, fix, or evaluate analytics tracking (GA4, GTM, product analytics, events, conversions, UTMs). This skill focuses on measurement strategy, signal quality, and validation— not just firing events.
Data Analysis
298superhero-tech/superhero-skills
Guide a full data-analyst workflow from business question to recommendation. Use when the user says "analyze this data/CSV", wants EDA on a fresh dataset, asks to find patterns in this dataset (segmentation, cohorts, correlations), or wants SQL queries over files or databases via DuckDB.
Browse the most popular Claude skills, all data & analytics skills, or write your own with the skills generator.