Deep learning techniques have achieved considerable success in pattern recognition and data classification tasks. However, existing Deep Neural Network (DNN) models are computationally expensive, limiting their effective utility in applications requiring performance guarantees in latency and throughput. The parallelism in DNN operations makes them ideal for modern GPU architectures. Nvidia's TensorRT tool aids in mapping neural network-based algorithms to Nvidia GPUs, promising resource optimization and inference acceleration. Given the closed nature of TensorRT's internal details, its effectiveness can only be evaluated empirically. With the rapid evolution of GPU hardware and software, periodic evaluations are necessary to validate findings, develop efficient methodologies, and verify the generality of techniques across different problems. This paper reports results obtained from a systematic experimental evaluation of TensorRT's optimization capabilities on GPUs of various ranges, from embedded systems to desktop computers, providing quantitative data on its advantages and limitations on both benchmark tasks and a real-world application. The findings offer guidelines for optimizing customized algorithms in scientific and industrial applications beyond standardized benchmarking.