Paste a Hugging Face model ID or pick one from the catalog, choose a quantization, context length and inference engine, and see how much VRAM it needs and which GPUs can run it, before you download tens of gigabytes.
Loading calculator…
How the calculation works
How is the VRAM needed by an LLM calculated?
We add five terms: the model weights, the KV cache, activation buffers, the inference framework overhead and the CUDA context. Weights depend on the parameter count and the quantization; the KV cache depends on the architecture (layers, KV heads, head size), the context length, the batch size and the cache precision.
Why does context length matter so much?
The KV cache grows linearly with every token of context and every concurrent sequence. For a 70B model with grouped-query attention, 128k tokens need about 40 GiB of cache on top of the weights. Sliding-window, MLA and hybrid layers shrink it, and the calculator models each of them.
How accurate is the estimate?
Weights and KV cache follow exact formulas. Activations and framework overhead are heuristic ranges that are not yet calibrated against measurements, so every result shows a minimum, expected and maximum value and the traffic light uses the maximum. Treat a yellow result as "may not fit".