Evaluating Llms Harness

lm-eval-harness: benchmark LLMs (MMLU, GSM8K, etc.).

9 stars 2 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add Lord1Egypt/RA-Skills --skill evaluating-llms-harness
Manual install
git clone --depth 1 https://github.com/Lord1Egypt/RA-Skills.git /tmp/RA-Skills
mkdir -p ~/.claude/skills
cp -r /tmp/RA-Skills/skills/built-in/mlops/evaluation/lm-evaluation-harness ~/.claude/skills/evaluating-llms-harness

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# lm-evaluation-harness - LLM Benchmarking

## What's inside

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

## Quick start

lm-evaluation-harness evaluates LLMs across 60+ academic benchmarks using standardized prompts and metrics.

**Installation**:
```bash
pip install lm-eval
```

**Evaluate any HuggingFace model**:
```bash
lm_eval --model hf \
  --model_args pretrained=meta-llama/Llama-2-7b-hf \
  --tasks mmlu,gsm8k,hellaswag \
  --device cuda:0 \
  --batch_size 8
```

**View available tasks**:
```bash
lm_eval --tasks list
```

## Common workflows

### Workflow 1: Standard benchmark evaluation

Evaluate model on core benchmarks (MMLU, GSM8K, HumanEval).

Copy this checklist:

```
Benchmark Evaluation:
- [ ] Step 1: Choose benchmark suite
- [ ] Step 2: Configure model
- [ ] Step 3: Run evaluation
- [ ] Step 4: Analyze results
```

**Step 1: Choose benchmark suite**

**Core reasoning benchmarks**:
- **MMLU** (Massive Multitask Language Understanding) - 57 subjects, multiple choice
- **GSM8K** - Grade school math word problems
- **HellaSwag** - Common sense reasoning
- **TruthfulQA** - Truthfulness and factuality
- **ARC** (AI2 Reasoning Challenge) - Science questions

**Code benchmarks**:
Read the full skill on GitHub

Details

Skill name
evaluating-llms-harness
Path
skills/built-in/mlops/evaluation/lm-evaluation-harness/SKILL.md
Repo updated
17 November 2026

FAQ

What does the Evaluating Llms Harness skill do?

lm-eval-harness: benchmark LLMs (MMLU, GSM8K, etc.).

How do I install the Evaluating Llms Harness skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add Lord1Egypt/RA-Skills --skill evaluating-llms-harness. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Evaluating Llms Harness popular?

It lives in Lord1Egypt/RA-Skills, which has 9 GitHub stars and 2 forks.

Related skills

Account Management

9

Lord1Egypt/RA-Skills

Reach the Plug&Pay Customer Portal (portal.plugandpay.com/login/{merchant-slug}) where a customer manages their account: view subscriptions, download invoices, edit billing details, change payment term, and cancel/resume a subscription. Documents the passwordless magic-link lo…

Documents & Office Files

Add Items to Cart

9

Lord1Egypt/RA-Skills

Resolve a free-text grocery list to specific King Soopers products (UPC, brand, size, price), ask follow-up questions for ambiguous items, then add the confirmed products to the signed-in user's cart for pickup or delivery.

Frontend & Design

Add to Cart

9

Lord1Egypt/RA-Skills

Find a product on kmart.com.au by natural-language query, navigate to the product detail page, capture title/price/SKU/availability, and drive the read-only pre-checkout flow up to the bag (/checkout/bag). Never submits an order — stops at the bag review screen. Documents the …

Other

Exa Web Search MCP

9

Lord1Egypt/RA-Skills

Connect an MCP client to Exa's hosted Search MCP server for web search, page fetching, and optional advanced search, with setup snippets for common clients, API-key handling, tool selection, and troubleshooting.

AI, LLMs & Agents

Create Monitor

9

Lord1Egypt/RA-Skills

Use the Parallel CLI to create and manage Monitors that continuously watch the web for new, material changes and notify you via webhook, polling, or Slack.

AI, LLMs & Agents

Exact Spec Packaging Procurement

9

Lord1Egypt/RA-Skills

Find exact-match Packrift packaging SKUs for a buyer requirement via the Packrift MCP, confirm live price and inventory, and return a measured cart or quote handoff without substituting nearby products as exact matches.

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.