Continuous LLM Red Teaming with Promptfoo

Wire Promptfoo and DeepTeam into CI/CD for automated regression red-teaming of LLM apps against OWASP LLM Top 10 and OWASP Agentic presets, failing the build when jailbreak or injection vulnerabilities regress.

100 stars 27 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add adriannoes/awesome-agentic-ai --skill continuous-llm-red-teaming-with-promptfoo
Manual install
git clone --depth 1 https://github.com/adriannoes/awesome-agentic-ai.git /tmp/awesome-agentic-ai
mkdir -p ~/.claude/skills
cp -r /tmp/awesome-agentic-ai/cursor-claude-codex/skills/anthropic-cybersecurity-skills/skills/continuous-llm-red-teaming-with-promptfoo ~/.claude/skills/continuous-llm-red-teaming-with-promptfoo

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Continuous LLM Red Teaming with Promptfoo

> **Authorized Use Only:** Run these adversarial probes only against LLM applications and endpoints you own or are explicitly authorized to test. Generated attack payloads (jailbreaks, prompt injections, harmful-content elicitation) are adversarial inputs; sending them to third-party services without permission may violate terms of service.

## Overview

Promptfoo is an open-source LLM evaluation and red-teaming framework (used by OpenAI and Anthropic per its README) that generates adversarial test cases, runs them against your model/agent, and grades the responses. DeepTeam (by Confident AI) is a complementary open-source framework offering 50+ ready-to-use vulnerabilities and 10+ research-backed attack methods. Together they let you treat LLM security as a **regression test**: every commit re-runs the same adversarial suite, and the pipeline fails when a previously-safe behavior regresses.
Read the full skill on GitHub

Details

Skill name
continuous-llm-red-teaming-with-promptfoo
Path
cursor-claude-codex/skills/anthropic-cybersecurity-skills/skills/continuous-llm-red-teaming-with-promptfoo/SKILL.md
License
MIT
Repo updated
2 November 2026

FAQ

What does the Continuous LLM Red Teaming with Promptfoo skill do?

Wire Promptfoo and DeepTeam into CI/CD for automated regression red-teaming of LLM apps against OWASP LLM Top 10 and OWASP Agentic presets, failing the build when jailbreak or injection vulnerabilities regress.

How do I install the Continuous LLM Red Teaming with Promptfoo skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add adriannoes/awesome-agentic-ai --skill continuous-llm-red-teaming-with-promptfoo. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Continuous LLM Red Teaming with Promptfoo popular?

It lives in adriannoes/awesome-agentic-ai, which has 100 GitHub stars and 27 forks, and is included in 2 curated skill lists.

Related skills

Browserstack

100

adriannoes/awesome-agentic-ai

Run tests on BrowserStack. Use when user mentions "browserstack", "cross-browser", "cloud testing", "browser matrix", "test on safari", "test on firefox", or "browser compatibility".

Testing & QAIn 2 lists27

Coverage

100

adriannoes/awesome-agentic-ai

Analyze test coverage gaps. Use when user says "test coverage", "what's not tested", "coverage gaps", "missing tests", "coverage report", or "what needs testing".

Testing & QAIn 2 lists27

Fix

100

adriannoes/awesome-agentic-ai

Fix failing or flaky Playwright tests. Use when user says "fix test", "flaky test", "test failing", "debug test", "test broken", "test passes sometimes", or "intermittent failure".

Testing & QAIn 2 lists27

Defending Llms with Guardrails

100

adriannoes/awesome-agentic-ai

Deploy Llama Guard, NeMo Guardrails, and LLM Guard input/output scanners as runtime defenses.

AI, LLMs & AgentsIn 2 lists27

Camada Agentica

100

adriannoes/awesome-agentic-ai

Use to propose and generate the project's agentic layer — rules (CLAUDE.md, settings.json), subagents (.claude/agents), skills (.claude/skills), and workflows/CI — tuned to the stack, tools, process, and domain. Proposes with justification and generates only what is approved. It is called by /kickoff and also runs standalone as the project evolves or new tools arrive. Trigger with /camada-agentica.

AI, LLMs & AgentsIn 2 lists27

Delegating to Agents

100

adriannoes/awesome-agentic-ai

How to delegate work to another AI agent (Pi, Codex, Claude Code, Hermes) — picking the right agent, sending prompts to TUI agents, polling progress. Read BEFORE any `cmux send`/`tmux send-keys` to an agent, or whenever delegating, relaying, spawning, or orchestrating agent-to-agent work.

AI, LLMs & AgentsIn 2 lists27

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.