Ability to blacklist certain packages from auto wheel building

[dstufft]

As of pip 7.0 pip will automatically build Wheels of sdists prior to installation, however some projects simply don't work when installed via Wheel. Before we release 7.0 we'll need some method for end users to opt out of, on a per project basis, the auto wheel building.

[rbtcollins]

I think we should have the blacklist apply to all wheels.

pip install --no-wheels-for=foo,bar,baz
-> don't accept wheels for them on indices
-> don't auto-build wheels for them

[dstufft]

That makes sense to me, I wonder if there's a (reasonable) way to make it more generic so you can explicitly list what formats you accept and don't accept for a particular project. In my head I can't think of a good way of doing that except for something like --(no|only)-wheels-for which is tied specifically into wheels vs sdist but that might be OK since I think it's unlikely we're ever going to support a second kind of binary distribution format and I can't think of a third type of package besides source and binary.

@pfmoore did raise an issue before about being able to ask for only wheels in #2084 which has some other discussion which might make sense to design something that works for both situation. He also called the option --binary-only so maybe --(no|only)-binary would make sense for a pair of options that take a list of project names that either cannot use binary packages or must use binary packages.

A third thought is that it might make sense for these options to warn if they are going to cause the latest version to be ignored because it's not a binary (or it's only a binary).

[rbtcollins]

How about:
--binary-only -> only binaries for everything
--no-binary -> no binaries for anything
--binary-only=x,y,z -> only binaries for those things
--no-binary=x,y,z -> no binaries for those things
with last-option-given-wins, and parameters are additive.
That is:
--binary-only --no-binary : will use no binaries for anything, won't autobuild wheels.
--binary-only --no-binary=lxml : will use binaries for everything except lxml(and won't autobuild wheels for it either)
--binary-only --no-binary=lxml,pyyaml : will use binaries for everything except lxml and pyyaml (and won't autobuild wheels for them either)

[rbtcollins]

W.r.t. logging a warning - I don't think we warn today do we (if someone has published only a wheel and no-use-wheel is in effect?)

[dstufft]

I like that, I'm not sure if our parsing can handle the case where something is sometimes a flag and sometimes not a flag (I guess you'd have to explicitly mandate that you use the = sign when it's not a flag but takes parameters?). I have a (not very) secret desire to eventually switch our commands over to using click instead of our hacked up home grown stuff. If our parser can handle --binary-only and --binary-only=foo being different we should double check that click's can as well. I am concerned that it would act differently than our other flags which support both --foo=bar and --foo bar.

I think that the option in general isn't going to be used a whole lot, and I think when it is that it's likely going to be used inside of a configuration file more often than it's used on the command line itself. What do you think about just using a special constant to define "all", like --binary-only :all: or --binary-only '*' or something?

WRT to a warning, I don't consider it a blocker to getting this issue done, I just thought it would be nice to issue a warning. I think we'd probably deprecate --[no-]-use-wheel in favor of this new option since it can handle every use case of that plus more, but you're correct we don't currently log a warning.

[rbtcollins]

We could use :all: if you like. I know argparse can do what I'm thinking of, and if click is build on argparse it should be able to (or should be extendable to be able to).

[dstufft]

pip (and click) are based on optparse.

[rbtcollins]

Alright, well I know optparse could do it too (hell, I did a top down syntax for arguments in optparse once :)) but doing :all: is easy enough.

[pfmoore]

Our options can also be specified in config files or environment variables, so you'd need to make sure that a "sometimes a flag" option worked there as well. I doubt that's possible, so I'd go for an :all: option.

On a bikeshedding note, I've seen programs use ALL (case sensitive) like this before, although I can't recall where offhand, but using :all: is new to me. Maybe ALL would be better for consistency? (With those programs I can't remember - this isn't a particularly good argument, is it? :-))

[rbtcollins]

ALL is a valid pypi package name.

[dstufft]

This is handled now in #2675

[znation]

@dstufft That link seems like a circular reference (#2675). Where can I find documentation on how to blacklist a package from auto wheel building?

[dstufft]

--no-binary package1,package2,package3

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.