NVIDIA TensorRT
developer.nvidia.com
Spec sheet
- Purpose
- TensorRT is an ecosystem of inference compilers, runtimes, and model optimization tools for high-performance deep learning inference.Read 5 Oct 2026 · developer.nvidia.com
- Security
- NVIDIA warns that deserializing an engine from an untrusted source is equivalent to running untrusted native code on the GPU and host.Read 5 Oct 2026 · docs.nvidia.com
- Cloud service access
- TensorRT Cloud is available with limited access to select partners, subject to approval.Read 5 Oct 2026 · developer.nvidia.com
- Deployment range
- TensorRT targets NVIDIA GPUs in data centers, workstations, laptops, and edge devices.Read 5 Oct 2026 · developer.nvidia.com
- Engine portability
- Serialized TensorRT engines are not portable across platforms such as Linux and Windows.Read 5 Oct 2026 · docs.nvidia.com
- Framework integrations
- TensorRT integrates with PyTorch and Hugging Face, imports ONNX models, and connects with MATLAB through GPU Coder.Read 5 Oct 2026 · developer.nvidia.com
- Hardware requirement
- The support matrix states that TensorRT supports NVIDIA hardware with compute capability SM 7.5 or higher.Read 5 Oct 2026 · docs.nvidia.com
- License limitation
- The SDK license says NVIDIA has not tested or certified the SDK for critical applications and places responsibility for applicable legal and regulatory compliance on the user.Read 5 Oct 2026 · docs.nvidia.com
- LLM inference
- TensorRT-LLM is an open-source library with a simplified Python API for accelerating and optimizing large language model inference on the NVIDIA AI platform.Read 5 Oct 2026 · developer.nvidia.com
- Optimization
- TensorRT optimizes inference with quantization, layer and tensor fusion, and kernel tuning.Read 5 Oct 2026 · developer.nvidia.com
- Security guidance
- NVIDIA recommends deserializing only engines built by the user or received through a trusted, authenticated channel.Read 5 Oct 2026 · docs.nvidia.com
- Serving
- NVIDIA Triton includes TensorRT as a backend and supports dynamic batching, concurrent model execution, model ensembling, and streaming audio and video inputs.Read 5 Oct 2026 · developer.nvidia.com
- Support resources
- NVIDIA provides TensorRT documentation, quick-start guides, sample code, and troubleshooting resources.Read 5 Oct 2026 · developer.nvidia.com
- Supported precisions
- TensorRT Model Optimizer supports FP8, FP4, INT8, INT4, and AWQ techniques.Read 5 Oct 2026 · developer.nvidia.com
Plans and prices
TensorRT Free Free for development · Download as a binary or NVIDIA NGC container · TensorRT 10.0 GA download requires NVIDIA Developer Program membership Read 5 Oct 2026 · developer.nvidia.com
NVIDIA AI Enterprise Priced on request Paid offering · Mission-critical AI inference · Enterprise-grade security, stability, manageability, and support Read 5 Oct 2026 · developer.nvidia.com
Where Specifiction places it
- Deep Learning Software#18 of 357.6