Data Distributed Storage

Use when the user asks about HDFS, S3, ADLS, GCS, MinIO or Ceph for data workloads, replication and erasure coding, small-file problems, storage tiering or lifecycle policies. Produces storage backend choice, layout and durability and cost settings. Not for database storage engines (use database-internals-mechanics) or file formats (use data-formats).

45 stars 6 forksDatabases

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add j4flmao/agent-skills --skill data-distributed-storage
Manual install
git clone --depth 1 https://github.com/j4flmao/agent-skills.git /tmp/agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/agent-skills/skills/data/distributed-storage ~/.claude/skills/data-distributed-storage

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Distributed Storage

## Purpose
Design distributed storage systems for data workloads. Select the right storage backend (HDFS, S3, ADLS, GCS, MinIO), configure durability and replication, optimize file formats and compression, and implement lifecycle policies.

## Agent Protocol

### Trigger
Exact user phrases: "HDFS", "S3", "ADLS", "GCS", "MinIO", "distributed storage", "object storage", "file system", "data lake storage", "storage tier", "lifecycle policy", "replication factor", "erasure coding", "storage cost".

### Input Context
- Storage backend preference (S3, ADLS, GCS, HDFS, MinIO)
- Data volume (current TB, annual growth)
- Access patterns (frequent, infrequent, archive)
- Durability and availability requirements
- Budget and cost constraints
- Compliance requirements (data residency, encryption)
- Network bandwidth and latency constraints

### Output Artifact
Distributed storage architecture with backend selection, tiering strategy, lifecycle policy, and cost model.

### Response Format
```yaml
# Storage backend
# Tier configuration
# Lifecycle rules
# Cost projection
```

### Completion Criteria
- [ ] Storage backend selected with rationale
- [ ] Storage tiers defined (hot/warm/cold/archive)
- [ ] Lifecycle policies configured for data movement
- [ ] Data durability and replication strategy documented
- [ ] Encryption (at rest, in transit) configured
- [ ] Cost projection for current and 3-year growth
- [ ] Backup and disaster recovery plan defined

## Workflow
Read the full skill on GitHub

Details

Skill name
data-distributed-storage
Path
skills/data/distributed-storage/SKILL.md
License
MIT
Repo updated
17 November 2026

FAQ

What does the Data Distributed Storage skill do?

Use when the user asks about HDFS, S3, ADLS, GCS, MinIO or Ceph for data workloads, replication and erasure coding, small-file problems, storage tiering or lifecycle policies. Produces storage backend choice, layout and durability and cost settings. Not for database storage engines (use database-internals-mechanics) or file formats (use data-formats).

How do I install the Data Distributed Storage skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add j4flmao/agent-skills --skill data-distributed-storage. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Data Distributed Storage popular?

It lives in j4flmao/agent-skills, which has 45 GitHub stars and 6 forks.

Related skills

Agent Observability

45

j4flmao/agent-skills

Use when the user asks about agent observability, tracing agent reasoning, debugging an agent loop, OpenTelemetry GenAI spans for LLM calls, LLM cost tracking per agent, agent audit logs or hallucination/eval monitoring in production. Produces instrumentation config, trace schema, dashboards, alerts and cost attribution. Not for generic service tracing without LLMs (use devops-opentelemetry).

DevOps & Cloud

AI Agent Architectures

45

j4flmao/agent-skills

Use when the user asks to build an AI agent, design an agent that calls tools and keeps memory, write an agent system prompt or pick an agent framework (Claude Agent SDK, OpenAI Agents SDK, LangGraph). Produces an architecture brief covering loop, tools, memory, guardrails and evals, routing depth to child skills. Not for workflow patterns (use ai-agentic-workflows).

AI, LLMs & Agents

AI Agent Core Loop

45

j4flmao/agent-skills

Use when the user asks about the agent loop, ReAct vs plan-and-solve, re-planning, Reflexion, an agent stuck in a loop, max iterations or termination conditions. Produces a loop spec or patch with state, success predicate, caps and no-progress detection. Not for overall agent design (use ai-agent-architectures) or tool schemas (use tool-calling-principles).

AI, LLMs & Agents

Data Graph Database

45

j4flmao/agent-skills

Use when the user asks about Neo4j and Cypher, Amazon Neptune, Gremlin, JanusGraph, SPARQL/RDF stores, graph traversal performance or knowledge graph storage. Produces graph model, queries and indexing and scaling choices. Not for relational/graph schema theory (use data-modeling) or GraphRAG prompting (use prompt-rag-query-design).

Databases

Data Distributed Compute

45

j4flmao/agent-skills

Use when the user asks about Spark vs Dask vs Ray, Spark executor sizing, shuffle and skew, YARN or Kubernetes resource management, data locality or speculative execution. Produces engine choice, cluster topology and tuning settings. Not for Hive/Spark SQL query tuning (use data-batch-processing) or GPU training (use distributed-training).

Databases

Data Lineage

45

j4flmao/agent-skills

Use when the user asks about data lineage, OpenLineage, Marquez, column-level lineage, impact analysis of a schema change or data provenance. Produces lineage collection setup (Airflow provider, dbt-ol, Spark listener), storage and impact queries. Not for catalog search and glossary (use data-data-catalog).

Databases

Browse the most popular Claude skills, all databases skills, or write your own with the skills generator.