Inference runtime
TensorRT-LLM
NVIDIA inference stack for optimized large-model serving on supported GPUs.
LinuxCUDATensorRT
Hardware
Supported hardware records
DGX B200 (8× B200)
NVIDIA
DGX Station (GB300)
NVIDIA
B300 SXM
NVIDIA
B200 SXM
NVIDIA · NVIDIA optimized inference stack.
GH200 Grace Hopper (144 GB HBM3e)
NVIDIA
H200 SXM
NVIDIA
DGX Spark (GB10)
NVIDIA
RTX PRO 6000 Blackwell Server Edition
NVIDIA
H100 NVL
NVIDIA
A100 SXM 80 GB
NVIDIA
H100 SXM
NVIDIA
L40S
NVIDIA