7
AirLLM Review 2026 — Run a 70B Model on a Single 4GB GPU (No Quantization, No Distillation)
AirLLM (27k stars, Apache-2.0) runs 70B LLMs on a single 4GB GPU with no quantization, distillation, or pruning — by streaming layers and per-expert shards from disk. It now handles DeepSeek-V3 671B on ~12GB and Kimi K3 2.8T on under 4GB. Full review of how it works, real measured speeds (including 292 s/token on Kimi K3), who it's actually for, and the HN skepticism.