Metric Validation Harness
Empirically validates a software metric before trusting or optimizing it — point it at any candidate metric (a command that takes a path and prints one number) plus a corpus, and it runs experiments that try to falsify each property a good metric must have. Checks determinism (same input, same number across runs and hash seeds), invariance to cosmetic edits (also an anti-gaming probe), monotonicity under construct-increasing edits, discrimination, robustness on edge inputs, near-linear tractability, and construct validity (convergent, discriminant vs LOC, predictive AUC, lift over a baseline).
Install
npx skills add pproenca/dot-skills --skill metric-validation-harnessgit clone --depth 1 https://github.com/pproenca/dot-skills.git /tmp/dot-skills
mkdir -p ~/.claude/skills
cp -r /tmp/dot-skills/skills/.experimental/metric-validation-harness ~/.claude/skills/metric-validation-harnessPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Metric Validation Harness Point this harness at a candidate metric and a corpus, and it runs experiments that try to **falsify** each property a trustworthy, optimizable metric must have. It is the empirical companion to `deterministic-metric-design`: that skill tells you to *prove* monotonicity, invariance, determinism, and construct validity; this skill *runs the experiment* and reports PASS/FAIL, each result mapped to the design-skill category it checks. **Read-only.** It computes and reports; it never modifies your metric, the corpus, or any external state. Safe to run unsupervised. ## When to Apply - Someone proposes, reviews, tunes, or ships a metric / score / index and you need evidence it is sound - A score "feels off" — you suspect it tracks LOC, jumps between runs, or saturates - You are about to let an agent **optimize** a metric and need to know it can't be gamed by cosmetic edits - You built a candidate per `deterministic-metric-design` and want to empirically confirm the properties you argued for - You are choosing between two metrics and need to know which actually predicts the outcome (and beats a trivial baseline) ## Workflow Overview
Details
- Skill name
metric-validation-harness- Repository
- pproenca/dot-skills
- Path
skills/.experimental/metric-validation-harness/SKILL.md- License
- MIT
- Repo updated
- 15 August 2026
FAQ
What does the Metric Validation Harness skill do?
Empirically validates a software metric before trusting or optimizing it — point it at any candidate metric (a command that takes a path and prints one number) plus a corpus, and it runs experiments that try to falsify each property a good metric must have. Checks determinism (same input, same number across runs and hash seeds), invariance to cosmetic edits (also an anti-gaming probe), monotonicity under construct-increasing edits, discrimination, robustness on edge inputs, near-linear tractability, and construct validity (convergent, discriminant vs LOC, predictive AUC, lift over a baseline).
How do I install the Metric Validation Harness skill?
skills CLI (Claude Code, Cursor, Codex and more): npx skills add pproenca/dot-skills --skill metric-validation-harness. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Metric Validation Harness popular?
It lives in pproenca/dot-skills, which has 228 GitHub stars and 19 forks.
Related skills
12 Factor App
228pproenca/dot-skills
The Twelve-Factor App methodology for building scalable, maintainable cloud-native applications. Use when designing backend services, APIs, microservices, or any software-as-a-service application. Triggers on deployment patterns, configuration management, process architecture, logging, and infrastructure decisions.
37signals Rails
228pproenca/dot-skills
37signals Rails coding principles and conventions from DHH, Jorge Manrubia, and the Fizzy/Basecamp/HEY codebase. This skill should be used when writing, reviewing, or refactoring Ruby on Rails code following the 37signals philosophy — vanilla Rails, CRUD controllers, rich domain models, concerns, no service objects, Hotwire, Turbo, Stimulus, Solid Queue, Solid Cache, Solid Cable, multi-tenancy, Minitest, custom auth, or DHH conventions.
Acceptance Pipeline Catalog
228pproenca/dot-skills
Use when implementing, reviewing, or debugging a Gherkin acceptance-test pipeline with mutation testing. Covers parser, JSON IR, generator, runtime, step handlers, test runner, mutator, value mutation rules, execution, result classification, reporting, project layout, conformance, and agent setup. Based on Uncle Bob's Acceptance Pipeline Specification. Trigger even when the user mentions Gherkin parsing, acceptance test generation, mutation testing for acceptance tests, or building a portable test pipeline.
Migrate Js to Modern Typescript
228pproenca/dot-skills
Migrating a JavaScript codebase to TypeScript — converting .js files to .ts, adding types to existing JS, or tightening a loosely-typed TS project toward strict mode. Covers tsconfig and allowJs strategy, incremental strict-flag ratcheting (noImplicitAny, strictNullChecks, noUncheckedIndexedAccess), typing public surfaces, replacing `any` and unsafe casts with `unknown` and narrowing, validating runtime boundaries (JSON, env, API responses), converting CommonJS to ESM and prototypes to classes, and the build/CI changes a migration needs. Trigger even when the user only says "add types", "turn
Linguistic Semantic Algorithms
228pproenca/dot-skills
Mapping out an unfamiliar codebase via NLP and graph algorithms — 40 algorithms across topic modelling, semantic embeddings, code graphs, repository mining, clone detection, IR-based bug localization, identifier linguistics, and complexity metrics. Trigger when hunting bugs across many files, scoping a new feature, identifying domain entities, or analyzing commit history — even if the user doesn't explicitly mention algorithms — apply when they ask "where does X live in this codebase?", "what is this codebase about?", "find duplicated logic", "what changed recently?", "who owns this code?", or
Npx CLI
191jwynia/agent-skills
Build and publish npx-executable CLI tools using Bun as the primary toolchain with npm-compatible output. Use when the user wants to create a new CLI tool, set up a command-line package for npx execution, configure argument parsing and terminal output, or publish a CLI to npm. Covers scaffolding, citty arg parsing, sub-commands, terminal UX, strict TypeScript, Biome + ESLint linting, Vitest testing, Bunup bundling, and publishing workflows. Keywords: npx, cli, command-line, binary, bin, tool, bun, citty, commander, terminal, publish, typescript, biome, vitest.
Browse the most popular Claude skills, all software development skills, or write your own with the skills generator.