Test Crewai Project
Deterministic, offline testing of CrewAI crews and flows with pytest: a stub BaseLLM (shipped in references/stub_llm.py) that drives text, structured output and tool calls with no API key, asserting on the prompts crewai built, testing guardrails and flow routing, what `crewai test` really does, event listeners for debugging, and the exact tracing/telemetry env var values. Use when writing tests for a crewai project, mocking or stubbing the LLM, writing a custom BaseLLM, seeing `TypeError: ... call() got an unexpected keyword argument 'from_task'`, `ValueError: OPENAI_API_KEY is required` in C
Install
/plugin marketplace add crewAIInc/skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add crewAIInc/skills --skill test-crewai-projectgit clone --depth 1 https://github.com/crewAIInc/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/skills/test-crewai-project ~/.claude/skills/test-crewai-projectPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Test a CrewAI Project
How to test crews and flows deterministically, without API keys, and how to debug real runs with events and traces.
Verified against crewai 1.15.22 and 1.15.23 on 2026-10-01.
Live-tested with real LLMs on 2026-10-01.
Run `crewai version` first; if the major/minor differs, re-verify the version-sensitive rows with the `ask-docs` skill.
---
## 1. Why tests need a stub LLM
A real LLM gives different text on every run, costs money, and needs a key. Every Agent without an explicit `llm` resolves to `gpt-4.1-mini` and fails at kickoff with `ValueError: OPENAI_API_KEY is required` when no key is set, so a CI run with no secrets cannot even start. You cannot assert on output you cannot predict.
A stub LLM fixes the responses, so assertions are exact. It keeps every real crewai code path: prompt building, `{placeholder}` interpolation, task context, tool execution, guardrail retries, pydantic conversion and flow routing. It also records every call, so you can assert on the prompts crewai built. That is where most bugs show up: a missing input, a dropped context, a tool that was never listed.
Test in tiers:
| Tier | Needs | What it proves |
|---|---|---|
| Unit: stub LLM, plain pytest | nothing | wiring, prompts, routing, parsing, guardrail logic |
| Guardrail truth tables | nothing | each guardrail accepts and rejects the right outputs |Details
- Skill name
test-crewai-project- Repository
- crewAIInc/skills
- Path
skills/test-crewai-project/SKILL.md- License
- MIT
- Repo updated
- 6 October 2026
FAQ
What does the Test Crewai Project skill do?
Deterministic, offline testing of CrewAI crews and flows with pytest: a stub BaseLLM (shipped in references/stub_llm.py) that drives text, structured output and tool calls with no API key, asserting on the prompts crewai built, testing guardrails and flow routing, what `crewai test` really does, event listeners for debugging, and the exact tracing/telemetry env var values. Use when writing tests for a crewai project, mocking or stubbing the LLM, writing a custom BaseLLM, seeing `TypeError: ... call() got an unexpected keyword argument 'from_task'`, `ValueError: OPENAI_API_KEY is required` in C
How do I install the Test Crewai Project skill?
Claude Code plugin: /plugin marketplace add crewAIInc/skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Test Crewai Project popular?
It lives in crewAIInc/skills, which has 49 GitHub stars and 24 forks.
Related skills
Ask Docs
49crewAIInc/skills
Query the official CrewAI documentation for answers. Use when the user has a CrewAI question that isn't fully covered by the getting-started, design-agent, design-task skills — e.g., specific API details, configuration options, advanced features, troubleshooting errors, enterprise features, tool references, or anything where the latest docs are the best source of truth.
Build Flow
49crewAIInc/skills
Building CrewAI Flows on crewai 1.15.x: structured (Pydantic) vs dict state and state.id, @start/@listen/@router wiring, or_/and_ semantics, router labels vs method names, @persist (correct import, where it stores state, restore_from_state_id), checkpointing with CheckpointConfig, @human_feedback and HumanFeedbackResult.feedback, plot(), calling crews and agents from flow methods, kickoff inputs, and the `crewai create flow` scaffold. Use when writing or debugging a Flow subclass or main.py with `from crewai.flow import ...`, when a listener never fires or fires twice, or when you see errors l
Call Deployed Crew
49crewAIInc/skills
Calling a crew or flow deployed on CrewAI AMP over HTTP: the bearer token and deployment URL, GET /inputs, POST /kickoff with the {"inputs": {...}} body, which input keys are required ({placeholder} tokens), polling GET /status/{kickoff_id} with a deadline and backoff, terminal states and where the result lives, webhooks, POST /resume, slow first calls, and writing crews that stay correct when they are kicked off repeatedly or concurrently. Use when writing a client, script, backend or frontend that calls a deployed crew, when a kickoff returns 422 "Missing inputs: ...", when a status poll nev
A B Testing
79LeadMagic/gtm-skills
Design statistically sound GTM experiments for copy, channels, landing pages, sequencing, pricing, and funnel conversion. Produces hypothesis, sample-size logic, success metrics, test plan, analysis method, and scale/stop/kill recommendation. Use when planning A/B tests, split tests, or experiment roadmaps.
Alterlab Test Selection Guard
87AlterLab-IEU/AlterLab-Academic-Skills
Enforces statistical-test selection as a discipline, holding the Iron Law NO TEST CHOSEN AFTER SEEING THE P-VALUE: routes the choice through a fixed decision tree (outcome type -> groups -> paired? -> normality) terminating in named tests (t-test, Mann-Whitney U, ANOVA, Kruskal-Wallis, Wilcoxon, Friedman, chi-square, Pearson/Spearman, regression), gates interpretation behind a mandatory Shapiro-Wilk/Levene/linearity assumption check, blocks test-shopping with an Excuse-vs-Reality table and Red-Flags-STOP list, and applies a 3+-tests escalation gate forcing Bonferroni/FDR correction or an explo
Lead Enrichment
79LeadMagic/gtm-skills
Enrich prospect and account lists with company, contact, role, technographic, intent, funding, hiring, and confidence fields. Produces enrichment schema, waterfall order, QA thresholds, dedupe rules, and CRM-ready output. Use when turning raw lists into usable GTM data or improving data quality.
Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.