FASTER Inference with Torch TensorRT Deep Learning for Beginners - CPU vs CUDA
Hi everyone! 😀 In the last video we've seen how to accelerate the speed of our programs with Pytorch and CUDA - today we will take it another step further with Torch-TensorRT!
We will focus on a Machine Learning process called Inference (which is when the model is trained, perfected and ready to make a prediction).
For this we will load a state-of-the-art artificial neural network and we will use it to classify a picture of my cat! 🙀🙀🙀
Specifically - we will borrow ResNet50 for our little Pytorch experiment! 😉
We will also run a speed test comparing Pytorch models running on CPU, on CUDA and on Torch-TensorRT - which of these do you think is faster??
⏲️ TIMESTAMPS ⏲️
-----------------------------------
00:00 - intro
01:05 - clone Torch-TensorRT
01:40 - install and setup Docker
03:52 - install Nvidia Container Toolkit & Nvidia Docker 2
05:02 - Torch-TensorRT container (option #1)
07:22 - Torch-TensorRT Nvidia NGC container (option #2)
09:00 - import Pytorch
09:16 - load ResNet50
10:25 - load sample image
11:45 - sample image transforms
14:48 - batch size
16:19 - prediction with ResNet50
17:12 - softmax function
18:07 - ImageNet class number to name mapping
20:10 - predict top 5 classes of sample image (topk)
23:33 - speed test benchmark function
27:33 - CPU benchmarks
28:13 - CUDA benchmarks
30:09 - trace model
31:20 - convert traced model into a Torch-TensorRT model
33:02 - TensorRT benchmarks
34:32 - download Jupyter Notebook
34:50 - HOW DID I MISS THIS???
35:31 - thanks for watching!
🛑 REFERENCED TUTORIALS 🛑
----------------------------------------------------------------------
⭐ CUDA Parallel Computing for beginners:
• CUDA Simply Explained - GPU vs CPU Pa...
⭐ Neural Networks for beginners:
• Neural Network Simply Explained - Dee...
⭐ Machine Learning Databases:
• Machine Learning Databases and How to...
⭐ Gradient Descent:
• Gradient Descent - Simply Explained! ...
⭐ INSTALLATION LINKS AND BASH COMMANDS ⭐
--------------------------------------------------------------------------------------
1. Clone Torch-TensorRT and change directory:
$ git clone https://github.com/NVIDIA/Torch-TensorRT
$ cd Torch-TensorRT
2. Docker installation guide:
https://docs.nvidia.com/datacenter/cl...
$ curl https://get.docker.com | sh \ && sudo systemctl --now enable docker
$ sudo groupadd docker
$ sudo usermod -aG docker $USER
$ newgrp docker
$ docker run hello-world
3. Nvidia Docker 2 installation:
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \ && curl -s -L https://nvidia.github.io/nvidia-docke... | sudo apt-key add - \ && curl -s -L https://nvidia.github.io/nvidia-docke... er.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
$ sudo apt-get update
$ sudo apt-get install -y nvidia-docker2
$ sudo systemctl restart docker
$ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
4. Official Torch TensorRT Container:
$ docker build -t torch_tensorrt -f ./docker/Dockerfile .
$ docker run --gpus=all --rm -it -v $PWD:/Torch-TensorRT --net=host --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 torch_tensorrt:latest bash
$ cd /Torch-TensorRT/notebooks
$ jupyter notebook --allow-root --ip 0.0.0.0 --port 8888
5. Nvidia NGC Container (alternative to 4):
https://catalog.ngc.nvidia.com/orgs/n...
$ docker pull nvcr.io/nvidia/pytorch:21.12-py3
$ docker run --net=host --gpus all -it --rm -v local_dir:/container_dir nvcr.io/nvidia/pytorch:21.12-py3
$ jupyter notebook --allow-root --ip 0.0.0.0 --port 8888
6. My cats picture:
https://github.com/MariyaSha/Inferenc...
7. Transforms Normalize - Pytorch Documentation:
https://pytorch.org/vision/stable/tra...
8. ImageNet class to name mapping:
https://github.com/pytorch/hub/blob/m...
9. Complete Notebook on Github:
https://github.com/MariyaSha/Inferenc...
---------------------------------------------------------------------------
💗 THANK YOU SO MUCH FOR WATCHING! 💗
Sound effect by: https://www.zapsplat.com
Icons by: https://www.flaticon.com/
Resnet50 image by: https://commons.wikimedia.org/wiki/Fi...
Watch video FASTER Inference with Torch TensorRT Deep Learning for Beginners - CPU vs CUDA online without registration, duration hours minute second in high quality. This video was added by user Python Simplified 01 January 1970, don't forget to share it with your friends and acquaintances, it has been viewed on our site 38,769 once and liked it 1 thousand people.