Test Crewai Project

Deterministic, offline testing of CrewAI crews and flows with pytest: a stub BaseLLM (shipped in references/stub_llm.py) that drives text, structured output and tool calls with no API key, asserting on the prompts crewai built, testing guardrails and flow routing, what `crewai test` really does, event listeners for debugging, and the exact tracing/telemetry env var values. Use when writing tests for a crewai project, mocking or stubbing the LLM, writing a custom BaseLLM, seeing `TypeError: ... call() got an unexpected keyword argument 'from_task'`, `ValueError: OPENAI_API_KEY is required` in C

49 stars 24 forksTesting & QA Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add crewAIInc/skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add crewAIInc/skills --skill test-crewai-project
Manual install
git clone --depth 1 https://github.com/crewAIInc/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/skills/test-crewai-project ~/.claude/skills/test-crewai-project

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Test a CrewAI Project

How to test crews and flows deterministically, without API keys, and how to debug real runs with events and traces.

Verified against crewai 1.15.22 and 1.15.23 on 2026-10-01.
Live-tested with real LLMs on 2026-10-01.
Run `crewai version` first; if the major/minor differs, re-verify the version-sensitive rows with the `ask-docs` skill.

---

## 1. Why tests need a stub LLM

A real LLM gives different text on every run, costs money, and needs a key. Every Agent without an explicit `llm` resolves to `gpt-4.1-mini` and fails at kickoff with `ValueError: OPENAI_API_KEY is required` when no key is set, so a CI run with no secrets cannot even start. You cannot assert on output you cannot predict.

A stub LLM fixes the responses, so assertions are exact. It keeps every real crewai code path: prompt building, `{placeholder}` interpolation, task context, tool execution, guardrail retries, pydantic conversion and flow routing. It also records every call, so you can assert on the prompts crewai built. That is where most bugs show up: a missing input, a dropped context, a tool that was never listed.

Test in tiers:

| Tier | Needs | What it proves |
|---|---|---|
| Unit: stub LLM, plain pytest | nothing | wiring, prompts, routing, parsing, guardrail logic |
| Guardrail truth tables | nothing | each guardrail accepts and rejects the right outputs |
Read the full skill on GitHub

Details

Skill name
test-crewai-project
Path
skills/test-crewai-project/SKILL.md
License
MIT
Repo updated
6 October 2026

FAQ

What does the Test Crewai Project skill do?

Deterministic, offline testing of CrewAI crews and flows with pytest: a stub BaseLLM (shipped in references/stub_llm.py) that drives text, structured output and tool calls with no API key, asserting on the prompts crewai built, testing guardrails and flow routing, what `crewai test` really does, event listeners for debugging, and the exact tracing/telemetry env var values. Use when writing tests for a crewai project, mocking or stubbing the LLM, writing a custom BaseLLM, seeing `TypeError: ... call() got an unexpected keyword argument 'from_task'`, `ValueError: OPENAI_API_KEY is required` in C

How do I install the Test Crewai Project skill?

Claude Code plugin: /plugin marketplace add crewAIInc/skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Test Crewai Project popular?

It lives in crewAIInc/skills, which has 49 GitHub stars and 24 forks.

Related skills

Ask Docs

49

crewAIInc/skills

Query the official CrewAI documentation for answers. Use when the user has a CrewAI question that isn't fully covered by the getting-started, design-agent, design-task skills — e.g., specific API details, configuration options, advanced features, troubleshooting errors, enterprise features, tool references, or anything where the latest docs are the best source of truth.

Writing & Content24

Build Flow

49

crewAIInc/skills

Building CrewAI Flows on crewai 1.15.x: structured (Pydantic) vs dict state and state.id, @start/@listen/@router wiring, or_/and_ semantics, router labels vs method names, @persist (correct import, where it stores state, restore_from_state_id), checkpointing with CheckpointConfig, @human_feedback and HumanFeedbackResult.feedback, plot(), calling crews and agents from flow methods, kickoff inputs, and the `crewai create flow` scaffold. Use when writing or debugging a Flow subclass or main.py with `from crewai.flow import ...`, when a listener never fires or fires twice, or when you see errors l

Writing & Content24

Call Deployed Crew

49

crewAIInc/skills

Calling a crew or flow deployed on CrewAI AMP over HTTP: the bearer token and deployment URL, GET /inputs, POST /kickoff with the {"inputs": {...}} body, which input keys are required ({placeholder} tokens), polling GET /status/{kickoff_id} with a deadline and backoff, terminal states and where the result lives, webhooks, POST /resume, slow first calls, and writing crews that stay correct when they are kicked off repeatedly or concurrently. Use when writing a client, script, backend or frontend that calls a deployed crew, when a kickoff returns 422 "Missing inputs: ...", when a status poll nev

DevOps & Cloud24

A B Testing

79

LeadMagic/gtm-skills

Design statistically sound GTM experiments for copy, channels, landing pages, sequencing, pricing, and funnel conversion. Produces hypothesis, sample-size logic, success metrics, test plan, analysis method, and scale/stop/kill recommendation. Use when planning A/B tests, split tests, or experiment roadmaps.

Testing & QAIn 1 list18

Alterlab Test Selection Guard

87

AlterLab-IEU/AlterLab-Academic-Skills

Enforces statistical-test selection as a discipline, holding the Iron Law NO TEST CHOSEN AFTER SEEING THE P-VALUE: routes the choice through a fixed decision tree (outcome type -> groups -> paired? -> normality) terminating in named tests (t-test, Mann-Whitney U, ANOVA, Kruskal-Wallis, Wilcoxon, Friedman, chi-square, Pearson/Spearman, regression), gates interpretation behind a mandatory Shapiro-Wilk/Levene/linearity assumption check, blocks test-shopping with an Excuse-vs-Reality table and Red-Flags-STOP list, and applies a 3+-tests escalation gate forcing Bonferroni/FDR correction or an explo

Testing & QAIn 1 list18

Lead Enrichment

79

LeadMagic/gtm-skills

Enrich prospect and account lists with company, contact, role, technographic, intent, funding, hiring, and confidence fields. Produces enrichment schema, waterfall order, QA thresholds, dedupe rules, and CRM-ready output. Use when turning raw lists into usable GTM data or improving data quality.

Testing & QAIn 1 list18

Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.