Find tools › Open-source projects › Keep data private

airllm
Open-source project · by lyogavin
Run massive open AI models on everyday GPUs with very little video memory by streaming layers from disk.
For developersPopular12,800 installs a month
What it costs
Free and open source; you may pay for the AI model it uses.
Use it to
- Run 70B parameter models on a single 4GB consumer GPU
- Perform local offline inference without renting expensive high-VRAM cloud instances
- Train or fine-tune massive models using layer-wise streaming
How to start
Install the library by running
pip install airllmin your Python environment.Import AutoModel and load a supported model identifier, optionally specifying 4bit or 8bit compression.
Tokenize your prompt and run model generation using standard PyTorch syntax.
Before you install
- Found automaticallyNot reviewed by a person yet. Check it before installing.
- License: Apache-2.0You are allowed to use it under these terms.
- Last updated 2026-11-18
More detail
AirLLM runs huge language models on consumer GPUs with very low VRAM by streaming and executing weights layer by layer.
Massive open models usually require tens or hundreds of gigabytes of GPU memory. AirLLM solves this by trading disk streaming speed for memory capacity, letting you run 70B+ models on small hardware.
Similar tools
Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-10-11. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map