AirLLM: How to Run Giant LLMs When Your GPU Is Way Too Small 🚀
What if your GPU has only 4 GB of VRAM, but the LLM you want to run needs tens or even hundreds of gigabytes? Normally, the answer sounds simple: You need a bigger GPU. But what if we don’t load the entire model into GPU memory in the first place…