Metric Validation Harness

Empirically validates a software metric before trusting or optimizing it — point it at any candidate metric (a command that takes a path and prints one number) plus a corpus, and it runs experiments that try to falsify each property a good metric must have. Checks determinism (same input, same number across runs and hash seeds), invariance to cosmetic edits (also an anti-gaming probe), monotonicity under construct-increasing edits, discrimination, robustness on edge inputs, near-linear tractability, and construct validity (convergent, discriminant vs LOC, predictive AUC, lift over a baseline).

228 stars 19 forksSoftware Development

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add pproenca/dot-skills --skill metric-validation-harness
Manual install
git clone --depth 1 https://github.com/pproenca/dot-skills.git /tmp/dot-skills
mkdir -p ~/.claude/skills
cp -r /tmp/dot-skills/skills/.experimental/metric-validation-harness ~/.claude/skills/metric-validation-harness

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Metric Validation Harness

Point this harness at a candidate metric and a corpus, and it runs experiments that try to **falsify** each property a trustworthy, optimizable metric must have. It is the empirical companion to `deterministic-metric-design`: that skill tells you to *prove* monotonicity, invariance, determinism, and construct validity; this skill *runs the experiment* and reports PASS/FAIL, each result mapped to the design-skill category it checks.

**Read-only.** It computes and reports; it never modifies your metric, the corpus, or any external state. Safe to run unsupervised.

## When to Apply

- Someone proposes, reviews, tunes, or ships a metric / score / index and you need evidence it is sound
- A score "feels off" — you suspect it tracks LOC, jumps between runs, or saturates
- You are about to let an agent **optimize** a metric and need to know it can't be gamed by cosmetic edits
- You built a candidate per `deterministic-metric-design` and want to empirically confirm the properties you argued for
- You are choosing between two metrics and need to know which actually predicts the outcome (and beats a trivial baseline)

## Workflow Overview
Read the full skill on GitHub

Details

Skill name
metric-validation-harness
Path
skills/.experimental/metric-validation-harness/SKILL.md
License
MIT
Repo updated
15 August 2026

FAQ

What does the Metric Validation Harness skill do?

Empirically validates a software metric before trusting or optimizing it — point it at any candidate metric (a command that takes a path and prints one number) plus a corpus, and it runs experiments that try to falsify each property a good metric must have. Checks determinism (same input, same number across runs and hash seeds), invariance to cosmetic edits (also an anti-gaming probe), monotonicity under construct-increasing edits, discrimination, robustness on edge inputs, near-linear tractability, and construct validity (convergent, discriminant vs LOC, predictive AUC, lift over a baseline).

How do I install the Metric Validation Harness skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add pproenca/dot-skills --skill metric-validation-harness. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Metric Validation Harness popular?

It lives in pproenca/dot-skills, which has 228 GitHub stars and 19 forks.

Related skills

12 Factor App

228

pproenca/dot-skills

The Twelve-Factor App methodology for building scalable, maintainable cloud-native applications. Use when designing backend services, APIs, microservices, or any software-as-a-service application. Triggers on deployment patterns, configuration management, process architecture, logging, and infrastructure decisions.

Software Development19

37signals Rails

228

pproenca/dot-skills

37signals Rails coding principles and conventions from DHH, Jorge Manrubia, and the Fizzy/Basecamp/HEY codebase. This skill should be used when writing, reviewing, or refactoring Ruby on Rails code following the 37signals philosophy — vanilla Rails, CRUD controllers, rich domain models, concerns, no service objects, Hotwire, Turbo, Stimulus, Solid Queue, Solid Cache, Solid Cable, multi-tenancy, Minitest, custom auth, or DHH conventions.

Software Development19

Acceptance Pipeline Catalog

228

pproenca/dot-skills

Use when implementing, reviewing, or debugging a Gherkin acceptance-test pipeline with mutation testing. Covers parser, JSON IR, generator, runtime, step handlers, test runner, mutator, value mutation rules, execution, result classification, reporting, project layout, conformance, and agent setup. Based on Uncle Bob's Acceptance Pipeline Specification. Trigger even when the user mentions Gherkin parsing, acceptance test generation, mutation testing for acceptance tests, or building a portable test pipeline.

Testing & QA19

Migrate Js to Modern Typescript

228

pproenca/dot-skills

Migrating a JavaScript codebase to TypeScript — converting .js files to .ts, adding types to existing JS, or tightening a loosely-typed TS project toward strict mode. Covers tsconfig and allowJs strategy, incremental strict-flag ratcheting (noImplicitAny, strictNullChecks, noUncheckedIndexedAccess), typing public surfaces, replacing `any` and unsafe casts with `unknown` and narrowing, validating runtime boundaries (JSON, env, API responses), converting CommonJS to ESM and prototypes to classes, and the build/CI changes a migration needs. Trigger even when the user only says "add types", "turn

Software Development19

Linguistic Semantic Algorithms

228

pproenca/dot-skills

Mapping out an unfamiliar codebase via NLP and graph algorithms — 40 algorithms across topic modelling, semantic embeddings, code graphs, repository mining, clone detection, IR-based bug localization, identifier linguistics, and complexity metrics. Trigger when hunting bugs across many files, scoping a new feature, identifying domain entities, or analyzing commit history — even if the user doesn't explicitly mention algorithms — apply when they ask "where does X live in this codebase?", "what is this codebase about?", "find duplicated logic", "what changed recently?", "who owns this code?", or

Software Development19

Npx CLI

191

jwynia/agent-skills

Build and publish npx-executable CLI tools using Bun as the primary toolchain with npm-compatible output. Use when the user wants to create a new CLI tool, set up a command-line package for npx execution, configure argument parsing and terminal output, or publish a CLI to npm. Covers scaffolding, citty arg parsing, sub-commands, terminal UX, strict TypeScript, Biome + ESLint linting, Vitest testing, Bunup bundling, and publishing workflows. Keywords: npx, cli, command-line, binary, bin, tool, bun, citty, commander, terminal, publish, typescript, biome, vitest.

Software Development24

Browse the most popular Claude skills, all software development skills, or write your own with the skills generator.