Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Support 'or' operators in marker expressions #82

Merged
merged 1 commit into from
Oct 2, 2019

Conversation

mladkau
Copy link
Contributor

@mladkau mladkau commented Oct 1, 2019

This is a triage fix for Zendesk #1672 and #1817. Supporting Environment Markers with expressions like:

futures==3.2.0; python_version == '2.6' or python_version >= "2.7"

@mladkau mladkau requested a review from a team as a code owner October 1, 2019 11:53
@ghost ghost requested review from dkontorovskyy and gitphill October 1, 2019 11:53
@CLAassistant
Copy link

CLAassistant commented Oct 1, 2019

CLA assistant check
All committers have signed the CLA.

@mladkau
Copy link
Contributor Author

mladkau commented Oct 1, 2019

I've added tests for matches_python_version and matches_environment and pointed out some bugs / unsupported features.

TBH I don't think the code is good enough for what we are trying to support here.

Trying to interpret a regular grammar like this with regular expressions will only work in a narrow subset of all possible cases.

Please do consider rewriting this code and implement a proper parser which could in the end support the majority of use cases and possible most environment markers (e.g. python_full_version, os_name, etc...) - a simple Top Down Operator Precedence parser (see here for an imperative example) should be enough for the expression grammar.

@@ -9,6 +9,7 @@
"main": "dist/index.js",
"scripts": {
"build": "tsc",
"watch": "tsc -w",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really related to the PR but I think this might be useful.

@@ -20,7 +20,7 @@
raise ImportError(
"Could not import pkg_resources; please install setuptools or pip.")

PYTHON_MARKER_REGEX = re.compile(r'python_version\s*(?P<operator>==|<=|=>|>|<)\s*\'(?P<python_version>.+)\'')
PYTHON_MARKER_REGEX = re.compile(r'python_version\s*(?P<operator>==|<=|=>|>|<)\s*[\'"](?P<python_version>.+?)[\'"]')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Supporting double quotes also for python markers and making the python_version group match non-greedy to avoid blowing up by attempting to convert 6\' or python_version >= \'2 into a number.

@@ -123,7 +123,7 @@ def create_package_as_root(package, dir_as_root):
dir_as_root[DEPENDENCIES][package_as_root[NAME]] = package_tree
return dir_as_root

def satisfies_python_requirement(parsed_operator, py_version_str, sys=sys):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's bad practice to pass in python modules as parameters. Better use mock for testing - keeps the shipped code clean and allows you to use proper contexts.

from collections import namedtuple

import unittest

fake_sys = namedtuple('Sys', ['version_info'])
from unittest.mock import patch
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume we can use python > 3.3 for the tests. For environment testing we should really use mock if at all possible.


# Gloss over the 'and' case and return true on the first matching python version

for sub_exp in re.split("\s*(?:and|or)\s*", cond_text):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We basically split up the expressions by or and and and return positive on the first matching python version.

@mladkau mladkau force-pushed the fix/support-or-statements branch 5 times, most recently from b8fd66c to c480f44 Compare October 1, 2019 12:58
@mladkau mladkau force-pushed the fix/support-or-statements branch from c480f44 to b3164cc Compare October 1, 2019 13:06
@@ -1,2 +1,3 @@
pipenv
virtualenv
mock
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding mock would be really useful for tests

@gitphill gitphill removed their request for review October 1, 2019 13:33
Copy link

@miiila miiila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for this fix. I agree we should probably take different and more general approach but I'm happy to approve this as a quick fix in order to improve the customer experience. Thank you for all the comments and very descriptive tests, it was very helpful.

@mladkau mladkau merged commit cf871d7 into master Oct 2, 2019
@mladkau mladkau deleted the fix/support-or-statements branch October 2, 2019 08:20
@snyksec
Copy link

snyksec commented Oct 2, 2019

🎉 This PR is included in version 1.13.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants