Fix bionicbuild image so we don't bundle two different pip version #104
Conversation
one which we install using get-pip. This should fix st2-packages dh-virtualenv step to use correct pip version.
This comment has been minimized.
This comment has been minimized.
This way it won't pull pip from system site packages and it will use the correct version.
|
OK, I managed to track down the root cause. The issue is that even though we install pip 20.0.2 everywhere, when dh virtualenv is creating virtualenv, it will use pip which comes with system Python distribution ( To fix this issue and unblock me on my various performance improvements PR, I will make a change to our
I believe that with #98 may not be needed anymore since virtualenv is also upgraded there which means it should use pip which ships with that version of venv - but as mentioned above, for now I just want to unblock myself and fix the build (and #98 is a slightly larger change). |
…d for --pip-version flag.
|
OK, I finally got it to work with dh-virtualenv changes from StackStorm/dh-virtualenv@1cf92b6 and also other changes from st2-packages PR in StackStorm/st2-packages#687. The whole thing is a bit of a madness. Even though we pin pip in various places (and specify I tested it locally and I can now build Bionic packages without issue and they use a correct pip version.
I also added some logging to that dh-virtualenv change. Without it, it was actually hard / impossible during build to see and assert that correct version was used. |
|
@armab Could I get access to last pass so I can see what is going on with Xenial Docker hub job? (I started failing even before I made any changes to Xenial Dockerfile). |
|
|
||
| # We use our dh-virtualenv version, since it fixes shebangd lines rewrites | ||
| RUN apt-get -y install \ | ||
| python-setuptools python-mock && \ | ||
| python-virtualenv python-setuptools python-mock && \ |
There was a problem hiding this comment.
Curious - why the move of python-virtualenv to here from earlier?
There was a problem hiding this comment.
That was done to make Bionic Dockerfile more similar to the Xenial one.
In fact, first I made more changes so those Dockerfiles were almost the same (also added ppa for Python so we install same versions - right now on Xenial we install Python 3.6.12 and on Bionic we install Python 3.6.9, etc.), but then I decided to remove those changes to keep the diff smaller.
If people feel opinionated about it, I can add them back (they work fine as well).
|
Dockerhub build issue is temporary apt mirror issue:
Work cast, to get myself unblocked I may just pin it to a fixed apt mirror in Dockerfile. |
This pull request fixes weird build issues I saw here - StackStorm/st2-packages#687 (comment).
The issue is that Bionic bundles two pip binaries before this change - system one installed via apt which is the old one and one we install using get-pip which is the correct / new one (20.0.2).
And during st2-packages build process, dh-virtualenv picks system one which won't work with new wheels, etc.
This pull request removes system one to ensure only a single pip (aka ltest 20.0.2 is available).
Now the whole Dockerfile also has left cruft and is more in line with Xenial one (which works fine since we don't install two pip versions tehre).