Deepeval

DeepEval is an open-source Python framework that unit-tests LLM applications with pytest-style test cases and LLM-as-a-judge metrics. Use when a user asks to evaluate a RAG chatbot, test LLM answers for relevancy, faithfulness or hallucination, write a custom metric, or run LLM quality checks in CI.

213 stars 34 forksTesting & QA

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add TerminalSkills/skills --skill deepeval
Manual install
git clone --depth 1 https://github.com/TerminalSkills/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/skills/deepeval ~/.claude/skills/deepeval

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# DeepEval — LLM Testing & Evaluation Framework

## Overview

DeepEval (4.3.1 as checked, October 2026) runs metrics over `LLMTestCase` objects, either under `deepeval test run` (a pytest wrapper) or through `evaluate()`. Most metrics call a judge LLM: `OPENAI_API_KEY` selects OpenAI by default, and `deepeval set-ollama`, `set-gemini` and similar commands switch the judge. Red teaming moved out into the separate DeepTeam package.

## Instructions

### Basic Test Cases

Write unit tests for LLM outputs using built-in metrics:
Read the full skill on GitHub

Details

Skill name
deepeval
Path
skills/deepeval/SKILL.md
License
Apache-2.0
Repo updated
8 November 2026

FAQ

What does the Deepeval skill do?

DeepEval is an open-source Python framework that unit-tests LLM applications with pytest-style test cases and LLM-as-a-judge metrics. Use when a user asks to evaluate a RAG chatbot, test LLM answers for relevancy, faithfulness or hallucination, write a custom metric, or run LLM quality checks in CI.

How do I install the Deepeval skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add TerminalSkills/skills --skill deepeval. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Deepeval popular?

It lives in TerminalSkills/skills, which has 213 GitHub stars and 34 forks.

Related skills

3dsmax Rendering

213

TerminalSkills/skills

Scripts and configures production rendering in Autodesk 3ds Max with the V-Ray and Corona renderers: output size and files, render elements, denoising, light mix, batch and command-line rendering, and network rendering. Use when a user asks to set up a production render, render several cameras in one batch, render from the command line or on a render farm, add render passes for compositing, or cut render time for archviz and product shots.

Other34

3dsmax Scripting

213

TerminalSkills/skills

Covers scripting Autodesk 3ds Max, the 3D modeling and rendering application, with MAXScript and Python (pymxs): scene manipulation, object creation, material assignment, camera and light setup, batch operations, and file I/O. Use when tasks involve automating repetitive 3ds Max workflows, batch processing scenes, running scripts headless with 3dsmaxbatch, creating custom tools, or scripting scene setup for archviz, product visualization, or VFX.

Productivity & Notes34

3proxy

213

TerminalSkills/skills

3proxy is a small open-source proxy server that runs HTTP/HTTPS, SOCKS4/5, SNI and TCP/UDP port-mapping proxies from one config file. Use when a user asks to set up an HTTP or SOCKS5 proxy, add proxy users and passwords, write 3proxy access rules, chain or rotate upstream (parent) proxies, limit bandwidth, connections or monthly traffic per user, run 3proxy in Docker, or fix a 3proxy.cfg that will not start.

DevOps & Cloud34

Design of Experiments

200

lyndonkl/claude

Generates structured experimental designs (factorial, response surface, Taguchi) to systematically discover how multiple factors affect outcomes while minimizing experimental runs. Use when optimizing multi-factor systems with limited experimental budget, screening many variables to find the vital few, discovering interactions between parameters, mapping response surfaces for peak performance, validating robustness to noise factors, or when users mention factorial designs, A/B/n testing, parameter tuning, or process optimization.

Testing & QA25

Cypress

213

TerminalSkills/skills

Assists with end-to-end testing of web applications using Cypress. Use when writing E2E tests, setting up component testing, configuring CI pipelines with parallelization, or building custom test commands. Trigger words: cypress, e2e testing, end-to-end, cypress run, cy.get, integration testing, browser testing.

Testing & QA34

Detox

213

TerminalSkills/skills

Detox is a gray-box end-to-end testing framework for React Native apps: it runs Jest tests against the app on an iOS simulator or Android emulator and waits for the app to go idle before each step. Use when the user wants to set up Detox, write or fix React Native E2E tests, debug a test that hangs or cannot find an element, or run Detox in CI. Also use when the user mentions "detox," "React Native E2E," "gray-box testing," "detox build," or "Wix Detox." For black-box testing of any mobile app, see appium; for simple YAML UI flows, see maestro.

Testing & QA34

Browse the most popular Claude skills, all testing & qa skills, or write your own with the skills generator.