Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Fix pip upgrade warning #6576

Merged
merged 4 commits into from
Aug 3, 2020

Conversation

theopolis
Copy link
Member

@theopolis theopolis commented Aug 2, 2020

Pip is warning us that a newer version is available.

Installing collected packages: psutil, timeout-decorator, six, thrift, argparse, future, osquery, pywin32
    Running setup.py install for timeout-decorator: started
    Running setup.py install for timeout-decorator: finished with status 'done'
    Running setup.py install for thrift: started
    Running setup.py install for thrift: finished with status 'done'
    Running setup.py install for future: started
    Running setup.py install for future: finished with status 'done'
Successfully installed argparse-1.4.0 future-0.18.2 osquery-3.0.6 psutil-5.7.2 pywin32-228 six-1.15.0 thrift-0.11.0 timeout-decorator-0.4.1
python.exe : WARNING: You are using pip version 20.1.1; however, version 20.2 is available.
At D:\a\_temp\e56a42ad-24b7-444f-a585-4f96f7940dbd.ps1:5 char:1
+ & $python3_path\python -m pip install setuptools psutil timeout_decor ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (WARNING: You ar...2 is available.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
##[error]PowerShell exited with code '1'.

One solution is to upgrade pip before using it.

@Smjert
Copy link
Member

Smjert commented Aug 2, 2020

Something I failed to mention in the past: I have configured Azure Pipelines to build master every 4 hours, to be able (if looking at it!) to catch these kind of failures earlier, or at least see if they were repeatable.
So if you have a look here https://dev.azure.com/trailofbits/osquery/_build?definitionId=6&branchFilter=50%2C50%2C50%2C50%2C50%2C50%2C50%2C50%2C50%2C50%2C50%2C50%2C50%2C50%2C50%2C50 (filtered by master branch) and on the left look for Scheduler for, you'll see that there's a lot of failures since Friday

@theopolis theopolis changed the title ci: Fix pip argparse warning ci: Fix pip upgrade warning Aug 3, 2020
@theopolis
Copy link
Member Author

@Smjert, this fixes the issue but it may not be the only (or best) method.

Copy link
Member

@directionless directionless left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine, if potentially slow. Dunno if you want to wait for @Smjert for it

Copy link
Member

@Smjert Smjert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this!

I did some tests and noticed that the situation changed when they updated the agent image version.
Python went from 3.8.4 to 3.8.5 but I don't think this is the issue because the warning about pip was already showing previously (although it's true that it's a NativeCommandError...).
Moreover I tested in a local VM with both Python versions, either directly inside powershell shell or a script and the process, despite the warning, never fails.
They must've changed something in the powershell task or in the powershell configuration.

The powershell task has a failOnStdErr that should have the task fail if something is written to stderr, although by default it's at false.
I tried on my Azure Pipeline instance to explicitly set it to false, but the failure is still there.

Long story short there's something strange going on, but I think we can wait to maybe better look into it; meanwhile this is a valid workaround, and it doesn't seem to slow down much, if at all.

@theopolis theopolis merged commit c08322b into osquery:master Aug 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants