diff --git a/docs/model_serving_framework/GPU_support.md b/docs/model_serving_framework/GPU_support.md index accda7ed36..e6e41a4faa 100644 --- a/docs/model_serving_framework/GPU_support.md +++ b/docs/model_serving_framework/GPU_support.md @@ -11,6 +11,7 @@ Test on AWS EC2 `g5.xlarge`, 64-bit(x86) - Ubuntu AMI: `Deep Learning AMI GPU PyTorch 1.12.1 (Ubuntu 20.04) 20221114` - Amazon Linux AMI: `Deep Learning AMI GPU PyTorch 1.12.1 (Amazon Linux 2) 20221114` - PyTorch: 1.12.1 +- CUDA: 11.6 ## 1.1 mount nvidia-uvm device @@ -60,6 +61,7 @@ Test on AWS EC2 `inf1.xlarge`, 64-bit(x86) - Ubuntu AMI: `Deep Learning AMI GPU PyTorch 1.12.1 (Ubuntu 20.04) 20221114` - Amazon Linux AMI: `Deep Learning AMI GPU PyTorch 1.12.1 (Amazon Linux 2) 20221114` - PyTorch: 1.12.1 +- CUDA: 11.6 ## 2.1 Fresh setup script @@ -430,6 +432,32 @@ For the second one run this sudo sysctl -w vm.max_map_count=262144 ``` +# 3. DOCKER + +Tested on AWS EC2 `g5.xlarge`, 64-bit(x86) + +- Amazon Linux AMI: `AWS Deep Learning Base AMI GPU CUDA 11 (Ubuntu 20.04) 20221104` +- PyTorch: 1.12.1 +- Docker: 20.10.21 +- CUDA: 11.6 +- CUDA Driver: 510.47.03 +- Docker Image: nvidia/cuda:11.6.2-cudnn8-devel-ubuntu20.04 + +Some example commands. + +Start nivida/cuda docker container: +``` +sudo sysctl -w vm.max_map_count=262144 +docker run -it --runtime=nvidia --gpus all -p 9200:9200 nvidia/cuda:11.6.2-cudnn8-devel-ubuntu20.04 /bin/bash +``` +Start OpenSearch in nivida/cuda docker container: +``` +wget https://artifacts.opensearch.org/releases/bundle/opensearch/2.5.0/opensearch-2.5.0-linux-x64.tar.gz +tar -xvf opensearch-2.5.0-linux-x64.tar.gz +cd opensearch-2.5.0 +bash opensearch-tar-install.sh +``` +