Will my model fit? A VRAM calculator
A model’s weights occupy roughly number of parameters × bytes per parameter. This is only a lower bound: inference also needs memory for the KV cache and activations, and training needs much more (gradients and optimizer state).
1 GB here means 109 bytes, matching how GPU memory is usually quoted. Example: 70 billion parameters × 2 bytes = 140 GB for the weights alone.
Terms
- VRAM
- The GPU’s own memory. If the model and its working data don’t fit in VRAM, you need more or bigger GPUs, or a smaller/quantised model.
- HBM and GDDR
- Two kinds of GPU memory. Data-centre GPUs such as the A100, H100, H200 and B200 use HBM (high-bandwidth memory); many workstation and gaming cards (RTX 4090, RTX 5090, L40S) use GDDR.
- SXM
- A socketed GPU module used on NVIDIA HGX server boards, where GPUs are linked to each other with NVLink. Listings such as “HGX H100” refer to this form factor.
- PCIe
- A GPU that is a standard add-in card. A PCIe variant of a GPU can differ from its SXM sibling in memory type, memory bandwidth and GPU-to-GPU connectivity, which is why this site lists them separately.
- NVLink
- NVIDIA’s high-speed GPU-to-GPU interconnect. It matters mainly when one job spans several GPUs.
- On-demand
- Pay as you go, start and stop any time. This is what the comparison shows.
- Spot / interruptible
- Cheaper capacity that the provider may take back. Suitable for work that can checkpoint and resume.
- Reserved / committed
- A lower rate in exchange for committing to a term or volume.
- Egress
- Data leaving the provider’s network. Some providers charge for it, some don’t; it is not part of the hourly GPU price.
- Billing granularity
- Whether time is billed per second, per minute or per hour. It matters for very short jobs.
Back to the price comparison.