Nanogpt Training

Train GPT-2 scale models (~124M parameters) efficiently on a single GPU. Covers GPT-124M architecture, tokenized dataset loading (e.g., HuggingFace Hub shards), modern optimizers (Muon, AdamW), mixed precision training, and training loop implementation.

49 stars 5 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add UCSB-NLP-Chang/Skill-Usage --skill nanogpt-training
Manual install
git clone --depth 1 https://github.com/UCSB-NLP-Chang/Skill-Usage.git /tmp/Skill-Usage
mkdir -p ~/.claude/skills
cp -r /tmp/Skill-Usage/tasks/mhc-layer-impl/environment/skills/nanogpt-training ~/.claude/skills/nanogpt-training

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# NanoGPT Training

## Overview

Training GPT-2 scale models (~124M parameters) efficiently on a single GPU. It provides:

- **GPT-124M Architecture**: Standard transformer with RoPE and modern optimizations
- **Tokenized Datasets**: Loading pre-tokenized shards from HuggingFace Hub or local files
- **Modern Optimizers**: Muon optimizer with Newton-Schulz orthogonalization
- **Mixed Precision**: bfloat16 training on A100 for 2x speedup

Training options:
- **Baseline GPT**: Standard residual connections
- **Experimental residual variants**: Optional alternative residual schemes for stability/efficiency

## Quick Reference

| Topic | Reference |
|-------|-----------|
| Model Architecture | [GPT Architecture](references/gpt-architecture.md) |
| Data Loading | [Tokenized Data](references/tokenized-data.md) |
| Optimizers | [Optimizers](references/optimizers.md) |
| Training Loop | [Training Loop](references/training-loop.md) |
| Hyperparameters | [Hyperparameters](references/hyperparameters.md) |

## Installation

```bash
pip install torch einops numpy huggingface_hub
```

## Minimal Example
Read the full skill on GitHub

Details

Skill name
nanogpt-training
Path
tasks/mhc-layer-impl/environment/skills/nanogpt-training/SKILL.md
Repo updated
8 April 2026

FAQ

What does the Nanogpt Training skill do?

Train GPT-2 scale models (~124M parameters) efficiently on a single GPU. Covers GPT-124M architecture, tokenized dataset loading (e.g., HuggingFace Hub shards), modern optimizers (Muon, AdamW), mixed precision training, and training loop implementation.

How do I install the Nanogpt Training skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add UCSB-NLP-Chang/Skill-Usage --skill nanogpt-training. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Nanogpt Training popular?

It lives in UCSB-NLP-Chang/Skill-Usage, which has 49 GitHub stars and 5 forks.

Related skills

Pubchem Database

49

UCSB-NLP-Chang/Skill-Usage

Query PubChem via PUG-REST API/PubChemPy (110M+ compounds). Search by name/CID/SMILES, retrieve properties, similarity/substructure searches, bioactivity, for cheminformatics.

Databases

Pymatgen

49

UCSB-NLP-Chang/Skill-Usage

Materials science toolkit. Crystal structures (CIF, POSCAR), phase diagrams, band structure, DOS, Materials Project integration, format conversion, for computational materials science.

Research & Science

Rdkit

49

UCSB-NLP-Chang/Skill-Usage

Cheminformatics toolkit for fine-grained molecular control. SMILES/SDF parsing, descriptors (MW, LogP, TPSA), fingerprints, substructure search, 2D/3D generation, similarity, reactions. For standard workflows with simpler interface, use datamol (wrapper around RDKit). Use rdkit for advanced control, custom sanitization, specialized algorithms.

Frontend & Design

Openai Vision

49

UCSB-NLP-Chang/Skill-Usage

Analyze images and multi-frame sequences using OpenAI GPT vision models

AI, LLMs & Agents

Modal GPU

49

UCSB-NLP-Chang/Skill-Usage

Run Python code on cloud GPUs using Modal serverless platform. Use when you need A100/T4/A10G GPU access for training ML models. Covers Modal app setup, GPU selection, data downloading inside functions, and result handling.

AI, LLMs & Agents

Seisbench Model API

49

UCSB-NLP-Chang/Skill-Usage

An overview of the core model API of SeisBench, a Python framework for training and applying machine learning algorithms to seismic data. It is useful for annotating waveforms using pretrained SOTA ML models, for tasks like phase picking, earthquake detection, waveform denoising and depth estimation. For any waveform, you can manipulate it into an obspy stream object and it will work seamlessly with seisbench models.

AI, LLMs & Agents

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.