Make cudatoolkit 11.x and cudnn 8.x the default major versions [WIP]

[rehno-lindeque]

Description of changes

This is still work-in-progress. (Needs checking)

• The desire to update the default cudnn was already mentioned in https://github.com/NixOS/nixpkgs/pull/164338/files/2000db2e332b552c3a7237b8fafb60be2c6687d2#r828723025 and cudnn: 8.3.0 -> 8.3.2 #164338 (comment)
• It should also help with pytorch-bin: 1.10.0 -> 1.11.0 #164712 which requires cudatoolkit >= 10.2.0

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.05 Release Notes (or backporting 21.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[samuela]

I'd say that this is ready to merge once nixpkgs-review and cuda-nix-testsuite are passing.

cc @NixOS/cuda-maintainers

[samuela]

This breaks the current pytorch version. According to my testing cudnn 7.6 with CUDA 10.2 is the latest version combo that I could get to work. But it sounds like upgrading to cudnn 8.3 is useful for the building latest pytorch? @rehno-lindeque Is there any documentation on the pytorch side of things about what versions of cuDNN/CUDA they support?

[rehno-lindeque]

It looks like CUDA 10.2 was technically already supposed to be the lower bound for pytorch v1.10

https://github.com/pytorch/pytorch/blob/v1.10.0/README.md#from-source

• NVIDIA CUDA 10.2 or above
• NVIDIA cuDNN v7 or above
• Compiler compatible with CUDA Note: You could refer to the cuDNN Support Matrix for cuDNN versions with the various supported CUDA, CUDA driver and NVIDIA hardwares

[rehno-lindeque]

But it sounds like upgrading to cudnn 8.3 is useful for the building latest pytorch? Is there any documentation on the pytorch side of things about what versions of cuDNN/CUDA they support?

I'm struggling a bit to find good information but these things seem relevant:

• https://github.com/pytorch/pytorch/blob/v1.10.0/README.md#install-dependencies

# CUDA only: Add LAPACK support for the GPU if needed
conda install -c pytorch magma-cuda110  # or the magma-cuda* that matches your CUDA version from https://anaconda.org/pytorch/repo

Pytorch 1.11.0 and 1.10.0 both have this in their documentation:

• https://github.com/pytorch/pytorch/blob/v1.10.0/README.md#building-the-image-yourself
• https://github.com/pytorch/pytorch/blob/v1.11.0/README.md#building-the-image-yourself

The Dockerfile is supplied to build images with CUDA 11.1 support and cuDNN v8.

So I think that suggests it should be possible to build with cudnn v8... 🤔

[samuela]

Interesting I wasn't able to get it to compile with any cuDNN v8.x version, but no need to worry about that now if we're going to be updating pytorch to v1.11 anyhow

[samuela]

If it continues to give issues compiling against versions that they claim are supported, we can open an upstream issue with the pytorch folks!

[rehno-lindeque]

Looks like nvidia does have a Download cuDNN v7.6.5 (November 18th, 2019), for CUDA 10.2.

But I'm going to keep pushing on this v8.3 upgrade for now and see if I can figure it out. (Just trying the pytorch 1.10 build against it now myself)

[rehno-lindeque]

Oh I think I finally understand now. It looks to me like probably the valid combinations (for pytorch) are

• cudnn 7.x + cuda 10.x, or
• cudnn 8.x + cuda 11.x

It seems pretty clear now looking at

• https://github.com/pytorch/pytorch/blob/v1.11.0/.circleci/docker/build.sh#L84-L262 and
• https://github.com/pytorch/pytorch/blob/v1.11.0/.circleci/cimodel/data/pytorch_build_definitions.py#L62

So a default bump to cudnn 8 would need to be paired with a default bump to cuda 11. Is that something we want to do? Or should I look at cudnn 7.6 with cudatoolkit 10.2 instead?

[samuela]

So a default bump to cudnn 8 would need to be paired with a default bump to cuda 11. Is that something we want to do? Or should I look at cudnn 7.6 with cudatoolkit 10.2 instead?

I'd say let's get the pytorch stuff merged first, and then worry about cudnn/cudatoolkit second. There's no need for pytorch to necessarily require the default cudnn/cudatoolkit versions. For better or worse it looks like 5446ad8 already circumvented a PR to update the pytorch source build to 1.11.0, but pytorch-bin will also require an update.

[FRidh]

This change is present on master.