Bio Machine Learning Model Validation

Validates predictive models on omics and biomedical data with nested cross-validation, group/batch/temporal-aware splits, the full data-leakage taxonomy, probability calibration, decision-curve net benefit, optimism correction, sample-size planning, and TRIPOD+AI reporting. Use when estimating model performance honestly, choosing a CV scheme, detecting leakage, or judging whether reported discrimination means the model is actually useful. For feature selection itself see machine-learning/biomarker-discovery; for confirmatory-trial inference see clinical-biostatistics/trial-reporting.

9 stars 3 forksAI, LLMs & Agents Plugin marketplace

Install

Claude Code plugin
/plugin marketplace add hackernoon/cuttlefish-agent-skills

The repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add hackernoon/cuttlefish-agent-skills --skill bio-machine-learning-model-validation
Manual install
git clone --depth 1 https://github.com/hackernoon/cuttlefish-agent-skills.git /tmp/cuttlefish-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/cuttlefish-agent-skills/plugins/data-science/skills/bio-machine-learning-model-validation ~/.claude/skills/bio-machine-learning-model-validation

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

## Version Compatibility

Reference examples tested with: numpy 1.26+, scikit-learn 1.4+ (note 1.6/1.8 API changes below).

Before using code patterns, verify installed versions match. If versions differ:
- Python: `pip show <package>` then `help(module.function)` to check signatures

scikit-learn drift to watch: `CalibratedClassifierCV(cv='prefit')` was deprecated in 1.6 and removed in 1.8 (it now raises; wrap a fitted model in `sklearn.frozen.FrozenEstimator` instead); `ensemble` default became `'auto'` in 1.6; `method='temperature'` was added in 1.8. If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

# Model Validation for Biomedical and Omics Data

**"Validate my omics classifier honestly"** -> Keep every data-dependent step inside the resampling loop, never use the same data to both choose and grade, and report calibration and net benefit, not just AUC.
- Nested CV: `GridSearchCV` (inner) wrapped by `cross_val_score` (outer)
- Group/structured: `StratifiedGroupKFold`, `TimeSeriesSplit`
- Calibration: `calibration_curve`, `brier_score_loss`, `CalibratedClassifierCV`

## The Single Most Important Modern Insight -- A Reported Number Is an Honest Estimate Only If Nothing Leaked and Nothing Was Graded on What Was Chosen
Read the full skill on GitHub

Details

Skill name
bio-machine-learning-model-validation
Path
plugins/data-science/skills/bio-machine-learning-model-validation/SKILL.md
License
MIT
Repo updated
4 November 2026

FAQ

What does the Bio Machine Learning Model Validation skill do?

Validates predictive models on omics and biomedical data with nested cross-validation, group/batch/temporal-aware splits, the full data-leakage taxonomy, probability calibration, decision-curve net benefit, optimism correction, sample-size planning, and TRIPOD+AI reporting. Use when estimating model performance honestly, choosing a CV scheme, detecting leakage, or judging whether reported discrimination means the model is actually useful. For feature selection itself see machine-learning/biomarker-discovery; for confirmatory-trial inference see clinical-biostatistics/trial-reporting.

How do I install the Bio Machine Learning Model Validation skill?

Claude Code plugin: /plugin marketplace add hackernoon/cuttlefish-agent-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Bio Machine Learning Model Validation popular?

It lives in hackernoon/cuttlefish-agent-skills, which has 9 GitHub stars and 3 forks.

Related skills

AI Traffic Tracking

9

hackernoon/cuttlefish-agent-skills

Compatibility pointer for AI assistant referrals and generative search visibility; maintained in Open Analytics.

AI, LLMs & Agents

Analytics Tracking

9

hackernoon/cuttlefish-agent-skills

Compatibility pointer for GA4 event, key-event and User-ID tracking; maintained in Open Analytics.

Data & Analytics

Bio Admet Prediction

9

hackernoon/cuttlefish-agent-skills

Predicts ADMET properties using ADMETlab 3.0 (119 platform features, including 77 prediction models with modeled-endpoint uncertainty), ADMET-AI, DeepChem MolNet, and chemprop D-MPNN with explicit handling of OECD QSAR principles, applicability domain assessment, calibration, hERG/CYP/AMES endpoints, and PAINS / Lipinski / Ro5 / Veber / BBB druglikeness filters. Use when filtering compounds for drug-likeness, prioritizing leads by predicted safety, or building an in-house ADMET QSAR model.

AI, LLMs & Agents

Bio Machine Learning Survival Analysis

9

hackernoon/cuttlefish-agent-skills

Builds and validates predictive time-to-event models on clinical and omics data with penalized Cox, random survival forests, gradient-boosted and deep survival models, and prediction-grade evaluation (Uno's C, time-dependent AUC, integrated Brier, calibration, competing risks). Use when building an individualized risk predictor or prognostic omics signature, choosing a survival model, or evaluating one beyond the C-index. For Kaplan-Meier, log-rank, and classical Cox hazard-ratio inference in a trial see clinical-biostatistics/survival-analysis.

AI, LLMs & Agents

Bio Machine Learning Biomarker Discovery

9

hackernoon/cuttlefish-agent-skills

Selects biomarker features from high-dimensional omics data using Boruta all-relevant selection, mRMR, LASSO/elastic-net, and stability selection, while controlling the leakage, irreproducibility, and correlated-feature traps that make most published signatures fail to replicate. Use when identifying candidate biomarkers, deciding between an all-relevant and a minimal-optimal selector, or judging whether a selected gene set is reproducible. For unbiased performance estimation of the resulting model see machine-learning/model-validation; for interpreting a trained model see machine-learning/pre

AI, LLMs & Agents

Bio Metabolomics Normalization Qc

9

hackernoon/cuttlefish-agent-skills

Designs QC, corrects signal drift, removes batch effects, filters features, normalizes samples, and imputes missing values for untargeted LC-MS/GC-MS metabolomics, framing each step as a measurement model that can create or erase biological signal. Use when processing a peak/feature table before statistical analysis, choosing a drift-correction or sample-normalization method, deciding QC RSD vs D-ratio filtering, or handling left-censored missing values. The feature table is produced by metabolomics/xcms-preprocessing or metabolomics/msdial-preprocessing; transformation/scaling for modeling de

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.