GPU cloud glossary

The terms you meet when comparing GPU rental prices, explained briefly.

Will my model fit? A VRAM calculator

A model’s weights occupy roughly number of parameters × bytes per parameter. This is only a lower bound: inference also needs memory for the KV cache and activations, and training needs much more (gradients and optimizer state).

1 GB here means 109 bytes, matching how GPU memory is usually quoted. Example: 70 billion parameters × 2 bytes = 140 GB for the weights alone.

Terms

VRAM
The GPU’s own memory. If the model and its working data don’t fit in VRAM, you need more or bigger GPUs, or a smaller/quantised model.
HBM and GDDR
Two kinds of GPU memory. Data-centre GPUs such as the A100, H100, H200 and B200 use HBM (high-bandwidth memory); many workstation and gaming cards (RTX 4090, RTX 5090, L40S) use GDDR.
SXM
A socketed GPU module used on NVIDIA HGX server boards, where GPUs are linked to each other with NVLink. Listings such as “HGX H100” refer to this form factor.
PCIe
A GPU that is a standard add-in card. A PCIe variant of a GPU can differ from its SXM sibling in memory type, memory bandwidth and GPU-to-GPU connectivity, which is why this site lists them separately.
NVLink
NVIDIA’s high-speed GPU-to-GPU interconnect. It matters mainly when one job spans several GPUs.
On-demand
Pay as you go, start and stop any time. This is what the comparison shows.
Spot / interruptible
Cheaper capacity that the provider may take back. Suitable for work that can checkpoint and resume.
Reserved / committed
A lower rate in exchange for committing to a term or volume.
Egress
Data leaving the provider’s network. Some providers charge for it, some don’t; it is not part of the hourly GPU price.
Billing granularity
Whether time is billed per second, per minute or per hour. It matters for very short jobs.

Back to the price comparison.