Large File

Use BEFORE reading any data file that could be large (CSV/TSV, Parquet, HDF5, FITS, NetCDF, NDJSON, genomics FASTQ/FASTA/VCF/BAM, GRIB, ROOT, or big text/simulation logs like VASP OUTCAR). Returns a compact memory pointer — header/schema/shape/sample/key numbers — by introspection and sampling in bounded memory, so you never load a file bigger than the context window into the model. Reference data via the pointer; read specific ranges deterministically.

2,105 stars 231 forksProductivity & Notes

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add ai4s-research/open-science --skill large-file
Manual install
git clone --depth 1 https://github.com/ai4s-research/open-science.git /tmp/open-science
mkdir -p ~/.claude/skills
cp -r /tmp/open-science/runtime/skills/core/large-file ~/.claude/skills/large-file

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Large files: reference, don't load

Scientific files routinely dwarf any context window (90 GB FASTQ, multi-GB
HDF5/FITS snapshots, 20 GB+ NetCDF rasters, huge VASP logs). Reading them raw
both OOMs and hallucinates — the materials case that consumed 20M+ tokens and
**failed** succeeded in ~1200 tokens with a memory-pointer approach.

**Rule: never `cat`/read a whole data file into your context.** Probe it first,
work from the returned pointer (schema + sample + key numbers), then read only
the specific rows/columns/ranges you need with the real library.

## Probe a file

The probe ships beside this SKILL.md. Run it on any data file **before** opening
it:

```bash
python "$XDG_CONFIG_HOME/opencode/skills/large-file/large_file_probe.py" DATA_FILE [--sample N]
```

It prints one compact JSON pointer on stdout — always tiny, regardless of file
size (a 13 MB CSV → ~800 bytes; a 16 MB HDF5 → ~450 bytes).

## What you get back

- **Tables (CSV/TSV)** — column names + inferred dtypes, approximate row count
  (streamed, constant memory), and a head **and** tail sample.
- **Parquet** — schema + row/column/row-group counts, from file metadata only
  (no column data read).
- **HDF5** — the dataset tree with shapes and dtypes (no array data read).
- **FITS** — HDU list with dimensions and header keys (memmapped headers).
- **NetCDF** — dimensions and variables with dtypes.
- **NDJSON** — union of keys, record count, and a sample.
Read the full skill on GitHub

Details

Skill name
large-file
Path
runtime/skills/core/large-file/SKILL.md
Repo updated
17 November 2026

FAQ

What does the Large File skill do?

Use BEFORE reading any data file that could be large (CSV/TSV, Parquet, HDF5, FITS, NetCDF, NDJSON, genomics FASTQ/FASTA/VCF/BAM, GRIB, ROOT, or big text/simulation logs like VASP OUTCAR). Returns a compact memory pointer — header/schema/shape/sample/key numbers — by introspection and sampling in bounded memory, so you never load a file bigger than the context window into the model. Reference data via the pointer; read specific ranges deterministically.

How do I install the Large File skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add ai4s-research/open-science --skill large-file. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Large File popular?

It lives in ai4s-research/open-science, which has 2,105 GitHub stars and 231 forks.

Related skills

Computer Use

2.1k

ai4s-research/open-science

Use when a task needs an app on the user's own screen — a native desktop application (instrument control software, Origin, GraphPad, ImageJ, a reference manager, a spreadsheet), or a desktop browser window that needs window-level control. Read before the first `computer` call. Covers the observe-act-observe loop, why element indexes go stale, which action to prefer for which control, what "unverified" means and why it must never be reported as success, and the recovery for each error code. Not for page-level web automation (use the browser connector) and not for anything that can be done from

Software Development231

Domain Check

2.1k

ai4s-research/open-science

Use ONLY when the user asks to check, validate, review or audit analysis code for scientific correctness, or when you are the reviewer agent running its checks. Never run it unprompted as part of ordinary analysis work. Covers physics, earth/geo, biology, chemistry, social science and bioprocess/fermentation. Runs a deterministic domain-correctness gate that catches code which runs but is scientifically wrong (unit/dimension mismatch, Euclidean distance on lat/lon without a CRS, 0-based/1-based coordinate and strand errors, impossible SMILES valence, uncorrected multiple comparisons, averaging

Research & Science231

Modal Run

2.1k

ai4s-research/open-science

Use when the user asks to run heavy or GPU work on Modal (the cloud compute platform) — writing a Modal function in the workspace, running it with the user's own `modal` CLI + token, and bringing results back. Data-to-compute for jobs too big for the laptop, without a Slurm cluster.

Writing & Content231

Hicks Law

3.4k

Owl-Listener/designer-skills

Apply Hick's Law — decision time grows with the number of simultaneous choices. Use when a screen offers too many options at once. For how many items survive in memory afterwards, use `millers-law`.

Productivity & Notes447

Traction Eos

2.7k

wondelai/skills

Implement the Entrepreneurial Operating System (EOS) to align vision and execution across a company. Use when the user mentions "EOS", "Entrepreneurial Operating System", "V/TO", "quarterly rocks", "Level 10 meetings", "accountability chart", "IDS process", "my company feels chaotic", "we keep having the same problems", or "get the whole team aligned". Also trigger when a growing company needs meeting structure, goal-setting frameworks, or a systematic way to solve recurring organizational issues. Covers the six EOS components: Vision, People, Data, Issues, Process, Traction. For team motivati

Productivity & NotesIn 2 lists266

Millers Law

3.4k

Owl-Listener/designer-skills

Apply Miller's Law — chunk information into groups of about four to fit working memory. Use when grouping fields, menu items, or steps. For reducing the number of choices offered, use `hicks-law`.

Productivity & Notes447

Browse the most popular Claude skills, all productivity & notes skills, or write your own with the skills generator.