Evaluating Llms Harness
lm-eval-harness: benchmark LLMs (MMLU, GSM8K, etc.).
Install
npx skills add Lord1Egypt/RA-Skills --skill evaluating-llms-harnessgit clone --depth 1 https://github.com/Lord1Egypt/RA-Skills.git /tmp/RA-Skills
mkdir -p ~/.claude/skills
cp -r /tmp/RA-Skills/skills/built-in/mlops/evaluation/lm-evaluation-harness ~/.claude/skills/evaluating-llms-harnessPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# lm-evaluation-harness - LLM Benchmarking ## What's inside Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs. ## Quick start lm-evaluation-harness evaluates LLMs across 60+ academic benchmarks using standardized prompts and metrics. **Installation**: ```bash pip install lm-eval ``` **Evaluate any HuggingFace model**: ```bash lm_eval --model hf \ --model_args pretrained=meta-llama/Llama-2-7b-hf \ --tasks mmlu,gsm8k,hellaswag \ --device cuda:0 \ --batch_size 8 ``` **View available tasks**: ```bash lm_eval --tasks list ``` ## Common workflows ### Workflow 1: Standard benchmark evaluation Evaluate model on core benchmarks (MMLU, GSM8K, HumanEval). Copy this checklist: ``` Benchmark Evaluation: - [ ] Step 1: Choose benchmark suite - [ ] Step 2: Configure model - [ ] Step 3: Run evaluation - [ ] Step 4: Analyze results ``` **Step 1: Choose benchmark suite** **Core reasoning benchmarks**: - **MMLU** (Massive Multitask Language Understanding) - 57 subjects, multiple choice - **GSM8K** - Grade school math word problems - **HellaSwag** - Common sense reasoning - **TruthfulQA** - Truthfulness and factuality - **ARC** (AI2 Reasoning Challenge) - Science questions **Code benchmarks**:
Details
- Skill name
evaluating-llms-harness- Repository
- Lord1Egypt/RA-Skills
- Path
skills/built-in/mlops/evaluation/lm-evaluation-harness/SKILL.md- Repo updated
- 17 November 2026
FAQ
What does the Evaluating Llms Harness skill do?
lm-eval-harness: benchmark LLMs (MMLU, GSM8K, etc.).
How do I install the Evaluating Llms Harness skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add Lord1Egypt/RA-Skills --skill evaluating-llms-harness. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Evaluating Llms Harness popular?
It lives in Lord1Egypt/RA-Skills, which has 9 GitHub stars and 2 forks.
Related skills
Account Management
9Lord1Egypt/RA-Skills
Reach the Plug&Pay Customer Portal (portal.plugandpay.com/login/{merchant-slug}) where a customer manages their account: view subscriptions, download invoices, edit billing details, change payment term, and cancel/resume a subscription. Documents the passwordless magic-link lo…
Add Items to Cart
9Lord1Egypt/RA-Skills
Resolve a free-text grocery list to specific King Soopers products (UPC, brand, size, price), ask follow-up questions for ambiguous items, then add the confirmed products to the signed-in user's cart for pickup or delivery.
Add to Cart
9Lord1Egypt/RA-Skills
Find a product on kmart.com.au by natural-language query, navigate to the product detail page, capture title/price/SKU/availability, and drive the read-only pre-checkout flow up to the bag (/checkout/bag). Never submits an order — stops at the bag review screen. Documents the …
Exa Web Search MCP
9Lord1Egypt/RA-Skills
Connect an MCP client to Exa's hosted Search MCP server for web search, page fetching, and optional advanced search, with setup snippets for common clients, API-key handling, tool selection, and troubleshooting.
Create Monitor
9Lord1Egypt/RA-Skills
Use the Parallel CLI to create and manage Monitors that continuously watch the web for new, material changes and notify you via webhook, polling, or Slack.
Exact Spec Packaging Procurement
9Lord1Egypt/RA-Skills
Find exact-match Packrift packaging SKUs for a buyer requirement via the Packrift MCP, confirm live price and inventory, and return a measured cart or quote handoff without substituting nearby products as exact matches.
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.