Continuous LLM Red Teaming with Promptfoo
Wire Promptfoo and DeepTeam into CI/CD for automated regression red-teaming of LLM apps against OWASP LLM Top 10 and OWASP Agentic presets, failing the build when jailbreak or injection vulnerabilities regress.
Install
npx skills add adriannoes/awesome-agentic-ai --skill continuous-llm-red-teaming-with-promptfoogit clone --depth 1 https://github.com/adriannoes/awesome-agentic-ai.git /tmp/awesome-agentic-ai
mkdir -p ~/.claude/skills
cp -r /tmp/awesome-agentic-ai/cursor-claude-codex/skills/anthropic-cybersecurity-skills/skills/continuous-llm-red-teaming-with-promptfoo ~/.claude/skills/continuous-llm-red-teaming-with-promptfooPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Continuous LLM Red Teaming with Promptfoo > **Authorized Use Only:** Run these adversarial probes only against LLM applications and endpoints you own or are explicitly authorized to test. Generated attack payloads (jailbreaks, prompt injections, harmful-content elicitation) are adversarial inputs; sending them to third-party services without permission may violate terms of service. ## Overview Promptfoo is an open-source LLM evaluation and red-teaming framework (used by OpenAI and Anthropic per its README) that generates adversarial test cases, runs them against your model/agent, and grades the responses. DeepTeam (by Confident AI) is a complementary open-source framework offering 50+ ready-to-use vulnerabilities and 10+ research-backed attack methods. Together they let you treat LLM security as a **regression test**: every commit re-runs the same adversarial suite, and the pipeline fails when a previously-safe behavior regresses.
Details
- Skill name
continuous-llm-red-teaming-with-promptfoo- Repository
- adriannoes/awesome-agentic-ai
- Path
cursor-claude-codex/skills/anthropic-cybersecurity-skills/skills/continuous-llm-red-teaming-with-promptfoo/SKILL.md- License
- MIT
- Repo updated
- 2 November 2026
FAQ
What does the Continuous LLM Red Teaming with Promptfoo skill do?
Wire Promptfoo and DeepTeam into CI/CD for automated regression red-teaming of LLM apps against OWASP LLM Top 10 and OWASP Agentic presets, failing the build when jailbreak or injection vulnerabilities regress.
How do I install the Continuous LLM Red Teaming with Promptfoo skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add adriannoes/awesome-agentic-ai --skill continuous-llm-red-teaming-with-promptfoo. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Continuous LLM Red Teaming with Promptfoo popular?
It lives in adriannoes/awesome-agentic-ai, which has 100 GitHub stars and 27 forks, and is included in 2 curated skill lists.
Related skills
Browserstack
100adriannoes/awesome-agentic-ai
Run tests on BrowserStack. Use when user mentions "browserstack", "cross-browser", "cloud testing", "browser matrix", "test on safari", "test on firefox", or "browser compatibility".
Coverage
100adriannoes/awesome-agentic-ai
Analyze test coverage gaps. Use when user says "test coverage", "what's not tested", "coverage gaps", "missing tests", "coverage report", or "what needs testing".
Fix
100adriannoes/awesome-agentic-ai
Fix failing or flaky Playwright tests. Use when user says "fix test", "flaky test", "test failing", "debug test", "test broken", "test passes sometimes", or "intermittent failure".
Defending Llms with Guardrails
100adriannoes/awesome-agentic-ai
Deploy Llama Guard, NeMo Guardrails, and LLM Guard input/output scanners as runtime defenses.
Camada Agentica
100adriannoes/awesome-agentic-ai
Use to propose and generate the project's agentic layer — rules (CLAUDE.md, settings.json), subagents (.claude/agents), skills (.claude/skills), and workflows/CI — tuned to the stack, tools, process, and domain. Proposes with justification and generates only what is approved. It is called by /kickoff and also runs standalone as the project evolves or new tools arrive. Trigger with /camada-agentica.
Delegating to Agents
100adriannoes/awesome-agentic-ai
How to delegate work to another AI agent (Pi, Codex, Claude Code, Hermes) — picking the right agent, sending prompts to TUI agents, polling progress. Read BEFORE any `cmux send`/`tmux send-keys` to an agent, or whenever delegating, relaying, spawning, or orchestrating agent-to-agent work.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.