Datahub Evals

Use this skill to run DataHub's saved evals and report answers for judging. Triggers on: "run our evals", "run the eval suite", "run eval urn:li:eval:...", "how are our evals doing", "check for eval regressions", "upload this answer as an eval result", "score this answer with the DataHub judge", "compare two agents on the same eval". Answers each eval in a fresh agent with the DataHub tools attached, reports the answer through the DataHub Cloud CLI, and reads back the verdict DataHub's own judge produced.

46 stars 98 forksAI, LLMs & Agents Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add datahub-project/datahub-skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add datahub-project/datahub-skills --skill datahub-evals
Manual install
git clone --depth 1 https://github.com/datahub-project/datahub-skills.git /tmp/datahub-skills
mkdir -p ~/.claude/skills
cp -r /tmp/datahub-skills/skills/datahub-evals ~/.claude/skills/datahub-evals

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# DataHub Evals

Run DataHub's saved evals, and report answers — yours or another agent's — for DataHub to
judge.

**You are the runner.** There is no script: you fetch the evals, answer each one in a fresh
agent, and report the answers. Every call to DataHub is one `evals` subcommand, so the queries
and the payload live in the CLI.

```bash
acryl-datahub-cloud evals --agent-context   # the CLI's own guide to its commands
```

Use `acryl-datahub-cloud evals`. A `datahub evals` form exists in the CLI's own help text and
in some notes, but the group is not wired into the `datahub` CLI in any shipped release — that
name answers `No such command 'evals'`, and this skill does not use it.

---

## Never simulate the judge

If DataHub's judge cannot be reached, **say so and stop.** Do not score the answer yourself,
do not ask a subagent to render a verdict "the way DataHub would", and do not present any
locally-produced score as a verdict.

A simulated verdict written in the house style reads as authoritative, gets pasted into a
comparison table, and is comparable with nothing. A model is also not a fair judge of an
answer it or a sibling produced.

That is why `--type` is never passed to `evals report`: omitting it routes the answer
through the same judge a native run gets, which is the only thing that makes two runs
comparable.

---

## Before you run anything

**The CLI is installed.**

```bash
acryl-datahub-cloud evals --help
```
Read the full skill on GitHub

Details

Skill name
datahub-evals
Path
skills/datahub-evals/SKILL.md
License
Apache-2.0
Repo updated
29 September 2026

FAQ

What does the Datahub Evals skill do?

Use this skill to run DataHub's saved evals and report answers for judging. Triggers on: "run our evals", "run the eval suite", "run eval urn:li:eval:...", "how are our evals doing", "check for eval regressions", "upload this answer as an eval result", "score this answer with the DataHub judge", "compare two agents on the same eval". Answers each eval in a fresh agent with the DataHub tools attached, reports the answer through the DataHub Cloud CLI, and reads back the verdict DataHub's own judge produced.

How do I install the Datahub Evals skill?

Claude Code plugin: /plugin marketplace add datahub-project/datahub-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Datahub Evals popular?

It lives in datahub-project/datahub-skills, which has 46 GitHub stars and 98 forks.

Related skills

Datahub Connector Planning

46

datahub-project/datahub-skills

Plans new DataHub connectors by classifying the source system, researching it using a dedicated agent or inline research, and generating a _PLANNING.md blueprint with entity mapping and architecture decisions. Use when building a new connector, researching a source system for DataHub, or designing connector architecture. Triggers on: "plan a connector", "new connector for X", "research X for DataHub", "design connector for X", "create planning doc", or any request to plan/research/design a DataHub ingestion source.

Productivity & Notes98

Datahub Connector PR Review

46

datahub-project/datahub-skills

Reviews DataHub connector implementations against 22 golden standards for compliance, code quality, silent failures, test coverage, type design, and merge readiness. Use when reviewing connector code, checking a PR, auditing a connector implementation, or verifying connector standards compliance.

Git & Code Review98

Datahub Enrich

46

datahub-project/datahub-skills

Use this skill when the user wants to add or update metadata in DataHub: descriptions, tags, glossary terms, ownership, deprecation, domains, data products, structured properties, documents, or field-level metadata. Triggers on: "add tag to X", "update description for X", "set owner of X", "add glossary term", "deprecate X", "create a domain", "create a glossary term", "add a document", or any request to modify DataHub metadata.

Other98

Vibe Upskill

43

bb-cccc/vibe-upskill

Extract coding knowledge from agent-assisted development sessions and convert them into interview-ready scenario Q&A cards and structured knowledge base documents. Use when the user wants to learn from their vibe coding history, prepare for technical interviews, build a personal knowledge base from agent workflows, review what an agent did during coding/debugging/deployment, or transform agent conversations into personal skills. Triggers: "vibe coding", "agent record", "knowledge extraction", "interview prep", "scenario questions", "what did the agent do", "learn from agent", "coding knowledge

AI, LLMs & Agents

Burn Token

43

shengxinjing/burn-token

Use when user wants to burn tokens aggressively - infinite loop of analysis, summarization, or heavy math computation. Maximizes token consumption by defeating KV cache, randomizing prompts, and dispatching verbose subagents. Supports time limits, round limits, or unlimited burn until interrupted.

AI, LLMs & Agents

Web Navigator

43

Plocr/Reasonix-computer-use

Web 浏览器自动化路由规则。通过 web_navigator 将 Web 场景路由到 Playwright MCP 或 mcp-chrome。

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.