Airllm

AirLLM runs very large open language models on a small GPU (or a CPU) by loading the model one layer at a time from disk instead of holding it all in memory, at the price of slow generation. Use when the user wants to run a 70B or larger Hugging Face model locally with 4-12 GB of VRAM, asks how to fit a model that does not fit in GPU memory, or says "airllm", "layer-by-layer inference", "run Llama 70B on a small GPU", "offline batch answers from a big model".

213 stars 34 forksAI, LLMs & Agents

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add TerminalSkills/skills --skill airllm
Manual install
git clone --depth 1 https://github.com/TerminalSkills/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/skills/airllm ~/.claude/skills/airllm

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# AirLLM — run models bigger than your GPU, one layer at a time

## Overview

AirLLM (`pip install airllm`) is a Python library that loads a Hugging Face causal language model, splits its weights into one file per layer on disk, and during generation brings a single layer into GPU memory, runs it, frees it and moves to the next (with the next layer prefetched on a background thread). GPU memory use is therefore set by the size of one layer, not the whole model: the project reports Llama-class 70B models in about 4 GB of VRAM without quantizing, and far larger mixture-of-experts models in more. The README's figures are VRAM only.

The trade-off is speed. Every generated token has to pass through every layer, so every token streams essentially the whole model from disk to the GPU. Expect seconds to tens of seconds per token for 70B-class models on a consumer SSD (the README gives no tokens-per-second figure, so measure on your hardware before promising anything). Use it for offline, non-interactive jobs where memory is the limit and latency is not: batch classification, overnight summarisation, one-off questions to a big model. It is not a chat server.

It is an inference library you call from Python, with a Hugging Face-style `generate`. It does not expose an OpenAI-compatible server.

## Instructions

### Install and check the machine
Read the full skill on GitHub

Details

Skill name
airllm
Path
skills/airllm/SKILL.md
License
Apache-2.0
Repo updated
8 November 2026

FAQ

What does the Airllm skill do?

AirLLM runs very large open language models on a small GPU (or a CPU) by loading the model one layer at a time from disk instead of holding it all in memory, at the price of slow generation. Use when the user wants to run a 70B or larger Hugging Face model locally with 4-12 GB of VRAM, asks how to fit a model that does not fit in GPU memory, or says "airllm", "layer-by-layer inference", "run Llama 70B on a small GPU", "offline batch answers from a big model".

How do I install the Airllm skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add TerminalSkills/skills --skill airllm. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Airllm popular?

It lives in TerminalSkills/skills, which has 213 GitHub stars and 34 forks.

Related skills

3dsmax Rendering

213

TerminalSkills/skills

Scripts and configures production rendering in Autodesk 3ds Max with the V-Ray and Corona renderers: output size and files, render elements, denoising, light mix, batch and command-line rendering, and network rendering. Use when a user asks to set up a production render, render several cameras in one batch, render from the command line or on a render farm, add render passes for compositing, or cut render time for archviz and product shots.

Other34

3dsmax Scripting

213

TerminalSkills/skills

Covers scripting Autodesk 3ds Max, the 3D modeling and rendering application, with MAXScript and Python (pymxs): scene manipulation, object creation, material assignment, camera and light setup, batch operations, and file I/O. Use when tasks involve automating repetitive 3ds Max workflows, batch processing scenes, running scripts headless with 3dsmaxbatch, creating custom tools, or scripting scene setup for archviz, product visualization, or VFX.

Productivity & Notes34

3proxy

213

TerminalSkills/skills

3proxy is a small open-source proxy server that runs HTTP/HTTPS, SOCKS4/5, SNI and TCP/UDP port-mapping proxies from one config file. Use when a user asks to set up an HTTP or SOCKS5 proxy, add proxy users and passwords, write 3proxy access rules, chain or rotate upstream (parent) proxies, limit bandwidth, connections or monthly traffic per user, run 3proxy in Docker, or fix a 3proxy.cfg that will not start.

DevOps & Cloud34

Anthropic SDK

213

TerminalSkills/skills

The Anthropic SDK (Python `anthropic`, TypeScript `@anthropic-ai/sdk`) calls Claude models through the Messages API. Use when a user asks to add Claude to an app, build a chatbot, stream Claude responses, give Claude tools (function calling), send images or PDFs, turn on adaptive thinking, count tokens, or pick a current Claude model id. Covers client setup, streaming, tool use, vision, thinking and effort, and current model ids and breaking changes.

AI, LLMs & Agents34

Aider

213

TerminalSkills/skills

Aider is an open-source terminal AI pair programmer that edits files in your git repository with an LLM and commits each change. Use when a user asks to run Aider, let an AI refactor or fix code from the command line, script Aider with --message, auto-run lint and tests after edits, or point Aider at Claude, GPT, Gemini, DeepSeek or a local model.

AI, LLMs & Agents34

Anthropics Official Pack

117

Peiiii/skild

All official Claude skills from anthropics/skills.

AI, LLMs & Agents10

Browse the most popular Claude skills, all ai, llms & agents skills, or write your own with the skills generator.