Dedupe Against Corpus
Checks a candidate seed against the existing substacker corpus (seeds, drafts, published) for exact duplicates (sha256 fingerprint) and near-duplicates (title Jaccard, first-200-word Jaccard, shared topic cluster). Exact match exits as SKIPPED. Near-match links via related_seeds rather than creating a duplicate. Use after topic tagging and density scoring, before writing the seed. Trigger keywords — dedupe, duplicate, already thought, near-match, related seed, fingerprint.
Install
/plugin marketplace add lyndonkl/claudeThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add lyndonkl/claude --skill dedupe-against-corpusgit clone --depth 1 https://github.com/lyndonkl/claude.git /tmp/claude
mkdir -p ~/.claude/skills
cp -r /tmp/claude/skills/dedupe-against-corpus ~/.claude/skills/dedupe-against-corpusPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
# Dedupe Against Corpus ## Table of Contents - [Three tiers of match](#three-tiers-of-match) - [Workflow](#workflow) - [Worked example](#worked-example) - [Guardrails](#guardrails) **Related skills:** Called by `ingest-inbox-item` step 4. Queried ad-hoc by `search-corpus`. Backlinks into matched seeds (the one place this skill writes outside new seeds). ## Three tiers of match 1. **Tier 1 — Exact fingerprint**: sha256 of body matches an existing seed's fingerprint. → `SKIPPED`. 2. **Tier 2 — Title similarity**: normalized Jaccard on title tokens ≥ 0.7. → `LINK` candidate. 3. **Tier 3 — Content similarity**: for seeds sharing ≥2 topic tags, first-200-word Jaccard ≥ 0.5. → `LINK` candidate. Tier 2 and 3 candidates are unioned (up to 3 total LINK targets). ## Workflow ``` Dedupe one candidate seed: - [ ] Step 1: Grep all corpus/**/*.md frontmatter for fingerprint match - [ ] Step 2: If exact match, return SKIPPED - [ ] Step 3: Normalized title Jaccard against all existing seeds - [ ] Step 4: For seeds sharing ≥2 topic tags, first-200-word Jaccard - [ ] Step 5: Union tier-2 and tier-3 candidates, cap at 3 - [ ] Step 6: If any LINK candidates, return LINK with related_seeds list; else CREATE - [ ] Step 7: For LINK, Edit matched seeds' related_seeds to add this candidate's id ``` ### Step 7: Backlink safely The matched seeds get their `related_seeds` field extended (append, never replace) ONLY IF: - `manual_edits: false` on the matched seed, OR
Details
- Skill name
dedupe-against-corpus- Repository
- lyndonkl/claude
- Path
skills/dedupe-against-corpus/SKILL.md- Repo updated
- 1 September 2026
FAQ
What does the Dedupe Against Corpus skill do?
Checks a candidate seed against the existing substacker corpus (seeds, drafts, published) for exact duplicates (sha256 fingerprint) and near-duplicates (title Jaccard, first-200-word Jaccard, shared topic cluster). Exact match exits as SKIPPED. Near-match links via related_seeds rather than creating a duplicate. Use after topic tagging and density scoring, before writing the seed. Trigger keywords — dedupe, duplicate, already thought, near-match, related seed, fingerprint.
How do I install the Dedupe Against Corpus skill?
Claude Code plugin: /plugin marketplace add lyndonkl/claude. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Dedupe Against Corpus popular?
It lives in lyndonkl/claude, which has 200 GitHub stars and 25 forks.
Related skills
Abstraction Concrete Examples
200lyndonkl/claude
Builds structured abstraction ladders that translate high-level principles into concrete, actionable examples across 3-5 levels. Bridges communication gaps, reveals hidden assumptions, and tests whether abstract ideas work in practice. Use when explaining concepts at different expertise levels, moving between abstract principles and concrete implementation, identifying edge cases by testing ideas against scenarios, designing layered documentation, decomposing complex problems into actionable steps, or bridging strategy-execution gaps.
Academic Letter Architect
200lyndonkl/claude
Guides the creation of evidence-based academic recommendation letters, reference letters, and award nominations that combine concrete examples, meaningful comparisons, and genuine enthusiasm. Use when writing recommendation letters for students, postdocs, or colleagues, or when user mentions recommendation letter, reference, nomination, letter of support, endorsement, or needs help with strong advocacy and comparative statements.
Adr Architecture
200lyndonkl/claude
Documents significant architectural and technical decisions with full context, alternatives considered, trade-offs analyzed, and consequences understood. Creates a decision trail that helps teams understand why decisions were made. Use when choosing between technology options, making infrastructure decisions, establishing standards, migrating systems, or when user mentions ADR, architecture decision, technical decision record, or decision documentation.
Documentation and Adrs
213TerminalSkills/skills
Writes architecture decision records (ADRs) and keeps the rest of a project's documentation true after a change: README, reference comments on public interfaces and the changelog. Use when a user says "write an ADR", "record this decision", "document why we chose X", "set up a decision log", "supersede the old decision", "update the README", "add a changelog entry" or "document this API". Produces records in the Nygard or MADR format, a Keep a Changelog entry, and a script that checks the log for gaps and broken supersession links.
Cross Poster Blurb
200lyndonkl/claude
Writes a 60-140 word third-person blurb for the Substack cross-post feature, positioned so another newsletter writer can paste it into their cross-post popup without editing. Third-person throughout ("In this piece, Kushal argues…"). No subscriber CTAs. Use as the cross-post arm of the Distribution Translator. Trigger keywords — cross-post, cross-poster, blurb, Substack cross-post, third person, positioning.
Docusaurus
213TerminalSkills/skills
Build documentation websites with Docusaurus. Use when a user asks to create a documentation site, build a developer portal, set up API docs, create a knowledge base, add versioned documentation, build a static docs site with search, create a blog alongside docs, or migrate from GitBook or ReadTheDocs. Covers project setup, docs structure, versioning, custom pages, blog, Algolia search, MDX components, and deployment.
Browse the most popular Claude skills, all writing & content skills, or write your own with the skills generator.