Bio Machine Learning Model Validation
Validates predictive models on omics and biomedical data with nested cross-validation, group/batch/temporal-aware splits, the full data-leakage taxonomy, probability calibration, decision-curve net benefit, optimism correction, sample-size planning, and TRIPOD+AI reporting. Use when estimating model performance honestly, choosing a CV scheme, detecting leakage, or judging whether reported discrimination means the model is actually useful. For feature selection itself see machine-learning/biomarker-discovery; for confirmatory-trial inference see clinical-biostatistics/trial-reporting.
Install
/plugin marketplace add hackernoon/cuttlefish-agent-skillsThe repo is a Claude Code plugin marketplace. Run /plugin afterwards to pick the plugin that includes this skill.
npx skills add hackernoon/cuttlefish-agent-skills --skill bio-machine-learning-model-validationgit clone --depth 1 https://github.com/hackernoon/cuttlefish-agent-skills.git /tmp/cuttlefish-agent-skills
mkdir -p ~/.claude/skills
cp -r /tmp/cuttlefish-agent-skills/plugins/data-science/skills/bio-machine-learning-model-validation ~/.claude/skills/bio-machine-learning-model-validationPersonal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.
From the SKILL.md
## Version Compatibility Reference examples tested with: numpy 1.26+, scikit-learn 1.4+ (note 1.6/1.8 API changes below). Before using code patterns, verify installed versions match. If versions differ: - Python: `pip show <package>` then `help(module.function)` to check signatures scikit-learn drift to watch: `CalibratedClassifierCV(cv='prefit')` was deprecated in 1.6 and removed in 1.8 (it now raises; wrap a fitted model in `sklearn.frozen.FrozenEstimator` instead); `ensemble` default became `'auto'` in 1.6; `method='temperature'` was added in 1.8. If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying. # Model Validation for Biomedical and Omics Data **"Validate my omics classifier honestly"** -> Keep every data-dependent step inside the resampling loop, never use the same data to both choose and grade, and report calibration and net benefit, not just AUC. - Nested CV: `GridSearchCV` (inner) wrapped by `cross_val_score` (outer) - Group/structured: `StratifiedGroupKFold`, `TimeSeriesSplit` - Calibration: `calibration_curve`, `brier_score_loss`, `CalibratedClassifierCV` ## The Single Most Important Modern Insight -- A Reported Number Is an Honest Estimate Only If Nothing Leaked and Nothing Was Graded on What Was Chosen
Details
- Skill name
bio-machine-learning-model-validation- Repository
- hackernoon/cuttlefish-agent-skills
- Path
plugins/data-science/skills/bio-machine-learning-model-validation/SKILL.md- License
- MIT
- Repo updated
- 4 November 2026
FAQ
What does the Bio Machine Learning Model Validation skill do?
Validates predictive models on omics and biomedical data with nested cross-validation, group/batch/temporal-aware splits, the full data-leakage taxonomy, probability calibration, decision-curve net benefit, optimism correction, sample-size planning, and TRIPOD+AI reporting. Use when estimating model performance honestly, choosing a CV scheme, detecting leakage, or judging whether reported discrimination means the model is actually useful. For feature selection itself see machine-learning/biomarker-discovery; for confirmatory-trial inference see clinical-biostatistics/trial-reporting.
How do I install the Bio Machine Learning Model Validation skill?
Claude Code plugin: /plugin marketplace add hackernoon/cuttlefish-agent-skills. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.
Is Bio Machine Learning Model Validation popular?
It lives in hackernoon/cuttlefish-agent-skills, which has 9 GitHub stars and 3 forks.
Related skills
AI Traffic Tracking
9hackernoon/cuttlefish-agent-skills
Compatibility pointer for AI assistant referrals and generative search visibility; maintained in Open Analytics.
Analytics Tracking
9hackernoon/cuttlefish-agent-skills
Compatibility pointer for GA4 event, key-event and User-ID tracking; maintained in Open Analytics.
Bio Admet Prediction
9hackernoon/cuttlefish-agent-skills
Predicts ADMET properties using ADMETlab 3.0 (119 platform features, including 77 prediction models with modeled-endpoint uncertainty), ADMET-AI, DeepChem MolNet, and chemprop D-MPNN with explicit handling of OECD QSAR principles, applicability domain assessment, calibration, hERG/CYP/AMES endpoints, and PAINS / Lipinski / Ro5 / Veber / BBB druglikeness filters. Use when filtering compounds for drug-likeness, prioritizing leads by predicted safety, or building an in-house ADMET QSAR model.
Bio Machine Learning Survival Analysis
9hackernoon/cuttlefish-agent-skills
Builds and validates predictive time-to-event models on clinical and omics data with penalized Cox, random survival forests, gradient-boosted and deep survival models, and prediction-grade evaluation (Uno's C, time-dependent AUC, integrated Brier, calibration, competing risks). Use when building an individualized risk predictor or prognostic omics signature, choosing a survival model, or evaluating one beyond the C-index. For Kaplan-Meier, log-rank, and classical Cox hazard-ratio inference in a trial see clinical-biostatistics/survival-analysis.
Bio Machine Learning Biomarker Discovery
9hackernoon/cuttlefish-agent-skills
Selects biomarker features from high-dimensional omics data using Boruta all-relevant selection, mRMR, LASSO/elastic-net, and stability selection, while controlling the leakage, irreproducibility, and correlated-feature traps that make most published signatures fail to replicate. Use when identifying candidate biomarkers, deciding between an all-relevant and a minimal-optimal selector, or judging whether a selected gene set is reproducible. For unbiased performance estimation of the resulting model see machine-learning/model-validation; for interpreting a trained model see machine-learning/pre
Bio Metabolomics Normalization Qc
9hackernoon/cuttlefish-agent-skills
Designs QC, corrects signal drift, removes batch effects, filters features, normalizes samples, and imputes missing values for untargeted LC-MS/GC-MS metabolomics, framing each step as a measurement model that can create or erase biological signal. Use when processing a peak/feature table before statistical analysis, choosing a drift-correction or sample-normalization method, deciding QC RSD vs D-ratio filtering, or handling left-censored missing values. The feature table is produced by metabolomics/xcms-preprocessing or metabolomics/msdial-preprocessing; transformation/scaling for modeling de
Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.