Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix missing import for UI and added windows support where tensorrt is not available #975

Merged
merged 5 commits into from
Dec 30, 2021

Conversation

DocGarbanzo
Copy link
Contributor

@DocGarbanzo DocGarbanzo commented Dec 27, 2021

Fix missing imports and supporting of windows where tensorrt is potentially not working

  1. Two imports that are used in the UI were not correctly added to the install files
  • pyyaml was missing in conda install files
  • pyyaml and plotly were missing in setup.py
  1. Bumped the superlinter to v4 because of issue: https://github.com/github/super-linter/issues/2253

  2. Also, moved tensorrt imports into function, so loading will not fail unless a tensorrt conversion is called. It seems windows 11 doesn't support tensorrt yet.

  3. Bumped version bugfix digit

… conversion function so the interpreter import doesn't fail on windows where trt is not fully supported.
@DocGarbanzo DocGarbanzo requested review from zlite and Ezward December 27, 2021 18:22
@Ezward
Copy link
Contributor

Ezward commented Dec 27, 2021

Doing Windows conda install using the branch, after successfully recreating the donkey conda environment and activating, my system does not find pip. I'm not sure if installing pip is an explicit part of the installation. I'm running the commands from the Windows Powershell.

conda update -n base -c defaults conda
conda env remove -n donkey
conda env create -f install\envs\windows.yml

That all succeeded, then:

PS C:\Users\emurm\projects\autorope\donkeycar> conda activate donkey
PS C:\Users\emurm\projects\autorope\donkeycar> pip install --user tensorflow==2.2.0
pip : The term 'pip' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if
a path was included, verify that the path is correct and try again.
At line:1 char:1
+ pip install --user tensorflow==2.2.0
+ ~~~
    + CategoryInfo          : ObjectNotFound: (pip:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException
```
It seems the issue is powershell.  If I use the legacy command line and conda activate donkey that worked and I was able to run pip.

So perhaps we need to explictly say to use the legacy command console and not powershell.

@Ezward
Copy link
Contributor

Ezward commented Dec 28, 2021

Attempting to install the version of tensorflow-gpu specified in the install instructions failes

(donkey) C:\Users\emurm\projects\autorope\donkeycar>conda install tensorflow-gpu==2.2.0
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - tensorflow-gpu==2.2.0

Current channels:

  - https://repo.anaconda.com/pkgs/main/win-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/win-64
  - https://repo.anaconda.com/pkgs/r/noarch
  - https://repo.anaconda.com/pkgs/msys2/win-64
  - https://repo.anaconda.com/pkgs/msys2/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

(donkey) C:\Users\emurm\projects\autorope\donkeycar>conda search tensorflow-gpu
Loading channels: done
# Name                       Version           Build  Channel
tensorflow-gpu                 1.8.0      h21ff451_0  pkgs/main
tensorflow-gpu                 1.9.0      hf154084_0  pkgs/main
tensorflow-gpu                1.10.0      hf154084_0  pkgs/main
tensorflow-gpu                1.11.0      h0d30ee6_0  pkgs/main
tensorflow-gpu                1.12.0      h0d30ee6_0  pkgs/main
tensorflow-gpu                1.13.1      h0d30ee6_0  pkgs/main
tensorflow-gpu                1.14.0      h0d30ee6_0  pkgs/main
tensorflow-gpu                1.15.0      h0d30ee6_0  pkgs/main
tensorflow-gpu                 2.0.0      h0d30ee6_0  pkgs/main
tensorflow-gpu                 2.1.0      h0d30ee6_0  pkgs/main
tensorflow-gpu                 2.3.0      he13fc11_0  pkgs/main
tensorflow-gpu                 2.5.0      h17022bd_0  pkgs/main
tensorflow-gpu                 2.6.0      h17022bd_0  pkgs/main

Attempting installation conda install tensorflow-gpu==2.3.0 fails with lot and lots of package conflicts

The fix here to install with pip pip install tensorflow-gpu==2.2.0, so we should change the docs to reflect this.

@DocGarbanzo
Copy link
Contributor Author

@Ezward - thanks for the thorough testing, that's quite useful to understand potential issues for windows users:

  1. pip is explicitly listed in the conda yaml file, so it will be installed in the donkey env. I have no idea why this wouldn't work in powershell, but I have experienced windows as highly non-predictable, so probably no big surprise. Anyway, the docs explicitly state one should run these commands out of the conda prompt, which in my experience opens a standard command prompt in windows, not a powershell, so we should be good.
  2. You are right we need to install tensorflow or tensorflow-gpu with pip. The docs are already reflecting this, so I think we should be good on that front, too.
  3. I'm checking why the linter CI falls over.... :-/

@DocGarbanzo
Copy link
Contributor Author

I've bumped the superlinter to version 4 as well, because we are hitting the bug described here: https://github.com/github/super-linter/issues/2253

Copy link
Contributor

@Ezward Ezward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for doing this. I think windows users will appreciate it. I'm a linux user, but I found it much easier to actually get the gpu working in windows than I did in linux.

@DocGarbanzo DocGarbanzo merged commit d5b9130 into dev Dec 30, 2021
@DocGarbanzo DocGarbanzo deleted the fix_windows_support_for_training branch December 30, 2021 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants