Normalizing Text for Measurement

Strip, parse, and segment a document before something else measures it — normalizers that fuse two blocks into one word because a close-token emits no separator, the two metric classes failing on different residue — character-based ones (reading time, token budgets, size limits) inflated by leftover syntax, word-based ones (word count, readability) corrupted by boundaries the strip never inserted — sibling consumers that skip the normalizer entirely, parser presets whose rule set silently reclassifies a construct, sections grouped by a non-unique heading key that collapse and double-count into

115 stars 18 forksOther Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add wdm0006/python-skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add wdm0006/python-skills --skill normalizing-text-for-measurement
Manual install
git clone --depth 1 https://github.com/wdm0006/python-skills.git /tmp/python-skills
mkdir -p ~/.claude/skills
cp -r /tmp/python-skills/skills/common/text-normalization ~/.claude/skills/normalizing-text-for-measurement

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Normalizing Text for Measurement

Any pipeline that reduces a document to a number — word count, reading time,
readability, a token budget, a dedup key, a checksum, a search index — has two
halves: a **normalizer** that turns markup into something comparable, and a
**measurer** that consumes it. Reviews concentrate on the measurer. Almost every
bug in this shape lives in the normalizer, and none of them raise.

The failure mode is uniform: the output is still a plausible string, the metric
is still a plausible number, and nothing anywhere reports a problem. See
[reporting-derived-metrics](../derived-metrics/SKILL.md) for the other end of the
same pipeline — what the measurer does when the sample is too small.

## A missing separator fuses two blocks into one token

A token-walking stripper emits text for content tokens and separators for close
tokens. Miss one close token and two blocks become one word.

```python
# Bad — paragraphs are separated, headings are not
for token in tokens:
    if token.type == "inline":
        out.append(token.content)
    elif token.type == "paragraph_close":
        out.append("\n\n")
```

`"# Title\n\nBody text here."` normalizes to `"TitleBody text here."`. One word
instead of four, and a word that appears in no dictionary. Every consumer
inherits it:

- word count drops by one per heading
- average word length rises, so readability scores report plain prose as far
Read the full skill on GitHub

Details

Skill name
normalizing-text-for-measurement
Path
skills/common/text-normalization/SKILL.md
License
MIT
Repo updated
18 November 2026

FAQ

What does the Normalizing Text for Measurement skill do?

Strip, parse, and segment a document before something else measures it — normalizers that fuse two blocks into one word because a close-token emits no separator, the two metric classes failing on different residue — character-based ones (reading time, token budgets, size limits) inflated by leftover syntax, word-based ones (word count, readability) corrupted by boundaries the strip never inserted — sibling consumers that skip the normalizer entirely, parser presets whose rule set silently reclassifies a construct, sections grouped by a non-unique heading key that collapse and double-count into

How do I install the Normalizing Text for Measurement skill?

Claude Code plugin: /plugin marketplace add wdm0006/python-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Normalizing Text for Measurement popular?

It lives in wdm0006/python-skills, which has 115 GitHub stars and 18 forks.

Related skills

Auditing Python Security

115

wdm0006/python-skills

Audits Python libraries for security vulnerabilities using Bandit, pip-audit, Semgrep, and detect-secrets. Identifies SQL injection, command injection, hardcoded credentials, secrets exposed through tracebacks, weak cryptography, and insecure deserialization. Use when reviewing library security, setting up security scanning in CI, or implementing secure coding patterns.

Security18

Building Accessible Interfaces

115

wdm0006/python-skills

Makes web and app UIs usable by keyboard, screen-reader, low-vision and motion-sensitive users — semantic HTML before ARIA, labelled controls, visible focus, WCAG AA contrast measured rather than eyeballed, touch targets, reduced motion, and automated axe checks plus a manual keyboard pass in CI or before every UI PR. Use when building or restyling a screen, adding a form, modal, menu or custom control, choosing colors, or reviewing a UI change.

Frontend & Design18

Building Browser Extensions

115

wdm0006/python-skills

Builds and tests Manifest V3 browser extensions — service-worker message channels that survive teardown, chrome.storage read-modify-write safety, settings validation in the popup, and a Jest chrome mock that can actually fail. Use when writing or reviewing a background service worker, a popup that reads/writes chrome.storage, an extension's Jest suite, or debugging an extension whose UI silently stops responding.

Software Development18

Swedish Payroll

105

erp-mafia/accounted-skills

Swedish payroll (lön & arbetsgivaravgifter) compliance reference. Covers AGI filing, sociala avgifter (31.42%), skatteavdrag/skattetabeller, förmånsbeskattning (bil/kost/friskvård), semesterlöneskuld, OB/övertid, traktamente, utlägg/milersättning, F-skatt/A-skatt, BAS 7xxx mapping, brutto/nettolöneavdrag, löneväxling, and sjuklön. Trigger on lön, payroll, AGI, arbetsgivaravgifter, skattetabell, förmåner, semesterlön, traktamente, F-skatt, sjuklön, löneväxling, BAS 7xxx, or booking salary/vacation pay/sick pay in a Swedish context. Always use over training data.

Other

微信渠道

49

little-thing/grok-wechat-plugin

用户要在 Grok Bot 里安装、登录、收发、登出或卸载微信 iLink 渠道时使用。入站私信经 webhook 唤醒并转交专属助手后,专属助手立刻 wechat_send 回复该微信对端。支持文字、语音转写、图片/视频/文件与正在输入。

Other10

Earnings Conf Call Sentiment

132

OctagonAI/skills

Analyze the overall sentiment and tone of management during earnings conference calls, including confidence levels, optimism indicators, and forward-looking language.

Other14

Browse the most popular Claude skills, all other skills, or write your own with the skills generator.