Directory
Hardware
Compare GPU/CPU chips, prompt-processing (prefill) speed and inference generation speed. Token rates are measured for specific models and runtimes, not theoretical hardware specifications.
29 hardware models · Showing 1–25
| Select hardware to compare | Hardware | GPU / CPU chip | Prefill (tok/s) | Inference (tok/s) | RAM / VRAM |
|---|---|---|---|---|---|
| Tesla V100 SXM3 32 GB | — | — | 32 GB GPU | ||
| Tesla V100 SXM2 16 GB | — | — | 16 GB GPU | ||
| Tesla P100 SXM2 16 GB | — | — | 16 GB GPU | ||
| T4 Tensor Core GPU | — | — | 16 GB GPU | ||
| RTX PRO 6000 Blackwell Workstation Edition | — | — | 96 GB GPU | ||
| RTX PRO 6000 Blackwell Server Edition | — | — | 96 GB GPU | ||
| RTX 6000 Ada Generation | — | — | 48 GB GPU | ||
| Radeon PRO W7900 | — | — | 48 GB GPU | ||
| Radeon AI PRO R9700 | — | — | 32 GB GPU | ||
| L40S | — | — | 48 GB GPU | ||
| L4 Tensor Core GPU | — | — | 24 GB GPU | ||
| MI355X | — | — | 288 GB GPU | ||
| Instinct MI350X | — | — | 288 GB GPU | ||
| Instinct MI325X | — | — | 256 GB GPU | ||
| Instinct MI300X | — | — | 192 GB GPU | ||
| H200 | — | — | 141 GB GPU | ||
| H100 | — | — | 80 GB GPU | ||
| H100 NVL | — | — | 94 GB GPU | ||
| GH200 Grace Hopper (144 GB HBM3e) | — | — | 144 GB GPU | ||
| GeForce RTX 5090 | — | — | 32 GB GPU | ||
| GeForce RTX 4090 | — | — | 24 GB GPU | ||
| GeForce RTX 3090 | — | — | 24 GB GPU | ||
| B300 | — | — | 288 GB GPU | ||
| B200 | — | — | 180 GB GPU | ||
| Arc Pro B70 | — | — | 32 GB GPU |