TensorRT

NVidia's TensorRT is a high-performance inference engine for deep learning models.

Installation

First ensure pip is up to date, and you also have installed wheel.

code.txt
1python3 -m pip install --upgrade pip
2python3 -m pip install wheel

Then install tensorrt with pip.

code.txt
1python3 -m pip install --upgrade tensorrt

Also

code.txt
1python3 -m pip install tensorrt-cu12 tensorrt-lean-cu12 tensorrt-dispatch-cu12

To run pre-built version compatible engines.

code.txt
1python3 -m pip install --upgrade tensorrt-lean
2python3 -m pip install --upgrade tensorrt-dispatch

Checking your installation

code.py
1import tensorrt
2print(tensorrt.__version__)
3assert tensorrt.Builder(tensorrt.Logger())