Data Formats
Use when the user asks about Parquet vs ORC vs Avro, Apache Arrow, Arrow Flight, row group size, encoding and compression (ZSTD, Snappy) or columnar vs row-oriented formats. Produces a format choice and writer settings for the access pattern. Not for table formats like Iceberg or Delta (use data-data-lake).
Install
npx skills add j4flmao/agent-skills --skill data-formatsgit clone --depth 1 https://github.com/j4flmao/agent-skills.git /tmp/agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/agent-skills/skills/data/data-formats ~/.claude/skills/data-formatsPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Data Formats ## Purpose Design efficient data storage and transfer using Apache Arrow, Parquet, Avro, and ORC formats with appropriate compression, schema evolution strategies, and high-performance transport via Arrow Flight. ## Agent Protocol ### Trigger Exact user phrases: "Apache Arrow", "Parquet", "Avro", "ORC", "Arrow Flight", "columnar", "row-oriented", "compression", "schema evolution", "file format", "data format", "columnar storage", "data serialization", "row group", "arrow table", "IPC format", "Flight SQL". ### Input Context Before activating, verify: - Data access patterns (full scan, column projection, row lookup, point queries) - Storage target (S3, HDFS, local disk, memory, network transfer) - Write patterns (append-heavy, overwrite partitions, streaming) - Processing framework (Spark, DuckDB, pandas, Polars, Dremio, ClickHouse) - Schema evolution requirements (add/drop/rename columns over time) - Compression requirements (storage cost vs CPU cost) ### Output Artifact Data format specification with file layout, encoding, compression, and schema configuration as SQL, YAML, and Python. ### Response Format ```sql -- Parquet DDL with encoding and compression ``` ```python -- Arrow table construction and IPC ``` ```yaml -- File format configuration for Spark/DuckDB ``` No preamble. No postamble. No explanations. No filler/hedging/transitions. Compress output. ### Completion Criteria - [ ] Format selection justified by access pattern and workload
Details
- Skill name
data-formats- Repository
- j4flmao/agent-skills
- Path
skills/data/data-formats/SKILL.md- License
- MIT
- Repo updated
- 17 November 2026
FAQ
What does the Data Formats skill do?
Use when the user asks about Parquet vs ORC vs Avro, Apache Arrow, Arrow Flight, row group size, encoding and compression (ZSTD, Snappy) or columnar vs row-oriented formats. Produces a format choice and writer settings for the access pattern. Not for table formats like Iceberg or Delta (use data-data-lake).
How do I install the Data Formats skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add j4flmao/agent-skills --skill data-formats. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Data Formats popular?
It lives in j4flmao/agent-skills, which has 45 GitHub stars and 6 forks.
Related skills
Agent Observability
45j4flmao/agent-skills
Use when the user asks about agent observability, tracing agent reasoning, debugging an agent loop, OpenTelemetry GenAI spans for LLM calls, LLM cost tracking per agent, agent audit logs or hallucination/eval monitoring in production. Produces instrumentation config, trace schema, dashboards, alerts and cost attribution. Not for generic service tracing without LLMs (use devops-opentelemetry).
AI Agent Architectures
45j4flmao/agent-skills
Use when the user asks to build an AI agent, design an agent that calls tools and keeps memory, write an agent system prompt or pick an agent framework (Claude Agent SDK, OpenAI Agents SDK, LangGraph). Produces an architecture brief covering loop, tools, memory, guardrails and evals, routing depth to child skills. Not for workflow patterns (use ai-agentic-workflows).
AI Agent Core Loop
45j4flmao/agent-skills
Use when the user asks about the agent loop, ReAct vs plan-and-solve, re-planning, Reflexion, an agent stuck in a loop, max iterations or termination conditions. Produces a loop spec or patch with state, success predicate, caps and no-progress detection. Not for overall agent design (use ai-agent-architectures) or tool schemas (use tool-calling-principles).
Dev Loop Readme Writer
45j4flmao/agent-skills
Use when the user asks to write a README for this repository, improve project documentation, add quick start, installation, usage or contributing sections, or badges. Produces a structured README.md. Not for API reference docs (use backend-openapi-documentation) or changelogs (use dev-loop-changelog-generator).
Data Data Lake
45j4flmao/agent-skills
Use when the user asks to choose between Iceberg, Delta Lake and Hudi, ACID tables on object storage, time travel, schema evolution, compaction, vacuum, Z-order, merge-on-read vs copy-on-write or data lake architecture. Produces table-format choice and table maintenance plan. Not for medallion and Unity Catalog platform design (use data-data-lakehouse).
Frontend Storybook
45j4flmao/agent-skills
Use when the user asks to document button variants or other components in Storybook, CSF 3 stories, args and controls, play functions, interaction or a11y tests in Storybook, Chromatic visual tests or Storybook config. Produces stories, docs and test setup for Storybook 9+. Not for general unit and E2E testing (use frontend-testing).
Browse the most popular Claude skills, all writing & content skills, or write your own with the skills generator.