Data Streaming

Use when the user asks to stream database changes with Debezium into Kafka, CDC, Kafka topic and partition design, Flink or Kafka Streams processing, exactly-once, watermarks or Kinesis. Produces streaming architecture, topic design and processing jobs with delivery guarantees. Not for batch pipelines (use data-etl-pipeline) or app messaging (use backend-message-queue).

45 stars 6 forksSoftware Development

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add j4flmao/agent-skills --skill data-streaming
Manual install
git clone --depth 1 https://github.com/j4flmao/agent-skills.git /tmp/agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/agent-skills/skills/data/streaming ~/.claude/skills/data-streaming

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Data Streaming

## Purpose
Design streaming data pipelines with Kafka topic architecture, Flink/Kafka Streams processing, schema evolution, and reliability guarantees.

## Agent Protocol

### Trigger
Exact user phrases: "streaming", "Kafka", "Flink", "Kinesis", "stream processing", "event stream", "real-time", "CDC", "change data capture", "message queue", "stream architecture", "Kafka topic", "Kafka consumer", "stream processing pipeline", "exactly-once", "schema registry", "Avro", "Debezium", "ksqlDB".

### Input Context
Before activating, verify:
- Streaming platform (Kafka, Kinesis, Pulsar)
- Processing framework (Flink, Kafka Streams, ksqlDB, Spark Streaming)
- Source systems (database CDC, application events, IoT, logs)
- Target systems (warehouse, lake, search index, cache)
- Throughput and latency requirements
- Data volume and retention requirements
- Security and compliance needs
- Team expertise with streaming technologies

### Output Artifact
Streaming pipeline design with topic model, processing logic, reliability config as YAML and SQL.

### Response Format
```yaml
# Topic topology
# Partition strategy
# Consumer group config
```
```sql
-- ksqlDB query
```
```java
// Flink job skeleton
```

No preamble. No postamble. No explanations. No filler/hedging/transitions. Compress output — why use many token when few do trick.

### Completion Criteria
- [ ] Topic model defined with partition count and retention
- [ ] Schema Registry configured with evolution rules
Read the full skill on GitHub

Details

Skill name
data-streaming
Path
skills/data/streaming/SKILL.md
License
MIT
Repo updated
17 November 2026

FAQ

What does the Data Streaming skill do?

Use when the user asks to stream database changes with Debezium into Kafka, CDC, Kafka topic and partition design, Flink or Kafka Streams processing, exactly-once, watermarks or Kinesis. Produces streaming architecture, topic design and processing jobs with delivery guarantees. Not for batch pipelines (use data-etl-pipeline) or app messaging (use backend-message-queue).

How do I install the Data Streaming skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add j4flmao/agent-skills --skill data-streaming. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Data Streaming popular?

It lives in j4flmao/agent-skills, which has 45 GitHub stars and 6 forks.

Related skills

Agent Observability

45

j4flmao/agent-skills

Use when the user asks about agent observability, tracing agent reasoning, debugging an agent loop, OpenTelemetry GenAI spans for LLM calls, LLM cost tracking per agent, agent audit logs or hallucination/eval monitoring in production. Produces instrumentation config, trace schema, dashboards, alerts and cost attribution. Not for generic service tracing without LLMs (use devops-opentelemetry).

DevOps & Cloud

AI Agent Architectures

45

j4flmao/agent-skills

Use when the user asks to build an AI agent, design an agent that calls tools and keeps memory, write an agent system prompt or pick an agent framework (Claude Agent SDK, OpenAI Agents SDK, LangGraph). Produces an architecture brief covering loop, tools, memory, guardrails and evals, routing depth to child skills. Not for workflow patterns (use ai-agentic-workflows).

AI, LLMs & Agents

AI Agent Core Loop

45

j4flmao/agent-skills

Use when the user asks about the agent loop, ReAct vs plan-and-solve, re-planning, Reflexion, an agent stuck in a loop, max iterations or termination conditions. Produces a loop spec or patch with state, success predicate, caps and no-progress detection. Not for overall agent design (use ai-agent-architectures) or tool schemas (use tool-calling-principles).

AI, LLMs & Agents

Deno

45

j4flmao/agent-skills

Use when the user asks about Deno 2, deno.json, deno task, Deno.serve, Deno KV, JSR imports, deno compile or permission flags. Produces Deno projects with least-privilege permissions, tests and deployment. Not for the Oak framework specifically (use oak-backend).

Software Development

Data Search Engine

45

j4flmao/agent-skills

Use when the user asks about Elasticsearch, OpenSearch or Solr: mappings and analyzers, BM25 relevance tuning, kNN hybrid search, aggregations, shard sizing, ILM or cluster operations. Produces index mappings, queries and cluster settings. Not for app-level search features (use backend-search-patterns) or vector DB choice (use data-vector-databases).

Software Development

Design Information Architecture

45

j4flmao/agent-skills

Use when the user asks about information architecture, sitemaps, navigation design, user flows, content hierarchy, labeling, taxonomy or card sorting results. Produces IA maps, navigation structures and flow diagrams. Not for running user research studies (use design-ux-research) or visual layout (use design-visual-design).

Software Development

Browse the most popular Claude skills, all software development skills, or write your own with the skills generator.