warn user when requested extra does not exist - 2138 #2142

derwolfe · 2014-11-26T21:28:18Z

Working on adding loggers for the pkg_resources.UnknonwExtra exception that can be thrown when a user attempts to install an unknown extra.

derwolfe · 2014-11-26T21:29:47Z

This is in reference to issue #2138

dstufft · 2014-11-26T21:38:49Z

I'm actually confused a little bit about how this is working right now. It appears that pip install requests[nonexistent] fails right now because it installs from a Wheel, but pip install --no-use-wheel requests[nonexistant] doesn't fail from an sdist (and doesn't print any kind of message either).

Since this code looks like it takes a failure and turns it into a warning I'm guessing that this is only going to take affect for Wheel installs. It would be great to unify this so that we raise the same warning in both cases.

derwolfe · 2014-11-26T21:47:26Z

Sorry about that dstufft, I meant to put a label on the PR that this was in progress. In effect, this PR doesn't do anything right now.

I mainly made the PR so that I could use travis.

msabramo · 2014-12-16T05:14:28Z

@derwolfe: Are you planning to update this?

derwolfe · 2014-12-16T12:19:27Z

@msabramo - yes, I am planning on working on it this week before Friday.

derwolfe · 2014-12-19T01:49:50Z

This is related to PR #2153. Once it lands, the job of this PR will (should) be very simple to implement. Most of the legwork is really done by the pkg_resources.Distribution.requires() function, which that PR enables for non-wheel installs.

derwolfe · 2014-12-20T22:05:33Z

Sine PR #2153 has landed, tbis should be pretty simple to implement. I'm on vacation now, but I will be able to start working on this again on Dec 28.

Need to access the extras defined on the distribution itself. This way I could just check for membership in the set of extras defined, and throw an error for each, instead of stopping entirely.

…t exisiting isn't handled

derwolfe · 2015-01-07T14:50:51Z

Still working on this, albeit very slowly. I've rebased against develop so the #2153 changes are included.

Currently, if a non existent option is passed in and a single exception is thrown, This means if a user has a list of options like

pip install twisted[zorro, tls]

where tls actually exists; because of zorro failing, the tls option will not be installed.

In my mind the correct behaviour should be: try to install every option, if the option doesn't exist, log it, and move on to the next. @dstufft: does this proposed behaviour seem reasonable to you?

derwolfe · 2015-02-26T13:19:54Z

Ready for review :-).

xavfernandez · 2015-02-26T14:43:44Z

pip/req/req_set.py

+                    for subreq in dist.requires(
+                            RequirementSet._available_extras(
+                                dist.extras,
+                                req_to_install.extras)):


The static methods seem a bit much for basically doing

set(req_to_install.extras) & set(dist.extras)

and

set(req_to_install.extras) - set(dist.extras)

I agree, I hadn't though about using sets. I will update the code to use this and will remove the static methods and unit tests covering them. Thanks for your feedback.

@xavfernandez - Though, there is an issue with sets not guaranteeing the order of their items. Would a decent compromise be to just inline the comprehensions? This would still remove the static methods and their unit tests.

You could always sort the set items if you need guaranteed ordering for tests or something.

True - I could just do sorted(missing). Thanks!

derwolfe · 2015-02-26T18:42:32Z

I've integrated @xavfernandez's suggested changes into the patch.
It looks like travis only failed due to a connection error installing dependencies.

If anyone can restart the build, it is ready for re-review.

sigmavirus24 · 2015-03-10T15:56:34Z

pip/req/req_set.py

+                    for missing in missing_requested:
+                        logger.warning(
+                            '%s does not provide the extra \'%s\'',
+                            dist, missing)


Mind if I ask for a bit of style consistency? You have

lval = func( args )

And

obj.method( args, more_args)

msabramo · 2015-03-11T07:12:27Z

tests/functional/test_install_extras.py

+        'simplewheel[nonexistant]', expect_stderr=True,
+    )
+    assert (
+        "simplewheel 2.0 does not provide the extra 'nonexistant'"


nonexistant => nonexistent

derwolfe · 2015-03-11T13:46:07Z

@msabramo - thanks for your comments. I've made the changes you requested.

rbu · 2015-03-12T18:25:57Z

Also fixes #852 ?

msabramo · 2015-03-12T19:39:12Z

Nice.

$ pip install 'bpython[nonexistent]'
Collecting bpython[nonexistent]
  Using cached bpython-0.14.1-py2.py3-none-any.whl
  bpython 0.14.1 does not provide the extra 'nonexistent'
...

$ pip install --no-use-wheel 'bpython[nonexistent]'
Requirement already satisfied (use --upgrade to upgrade): bpython[nonexistent] in /Users/marca/python/virtualenvs/pip/lib/python2.7/site-packages
  bpython 0.14.1 does not provide the extra 'nonexistent'

It's nice that it's a warning instead of a fatal error with a traceback.

I did run into a problem where it couldn't figure out the name of the package when I tried it with sentry (the example in #852):

$ pip install -U 'sentry[lol]'
...
  Unknown 7.4.1 does not provide the extra 'lol'

Note how it say "Unknown" instead of "sentry".

Although to be fair, the current code on develop has the same problem:

$ pip install -U 'sentry[lol]'
Collecting sentry[lol]
  Using cached sentry-7.4.1.tar.gz
  Exception:
  Traceback (most recent call last):
    File "/Users/marca/dev/git-repos/pip/pip/basecommand.py", line 232, in main
      status = self.run(options, args)
    File "/Users/marca/dev/git-repos/pip/pip/commands/install.py", line 342, in run
      requirement_set.prepare_files(finder)
    File "/Users/marca/dev/git-repos/pip/pip/req/req_set.py", line 427, in prepare_files
      req_to_install.extras):
    File "/Users/marca/dev/git-repos/pip/pip/_vendor/pkg_resources/__init__.py", line 2504, in requires
      "%s has no such extra feature %r" % (self, ext)
  UnknownExtra: Unknown 7.4.1 has no such extra feature 'lol'

msabramo · 2015-03-12T19:42:35Z

$ pip install -U 'sentry[lol]'
Collecting sentry[lol]
  Using cached sentry-7.4.1.tar.gz
processing ~/.pdbrc.py
[5] > /Users/marca/dev/git-repos/pip/pip/_vendor/pkg_resources/__init__.py(2504)requires()
-> raise UnknownExtra(
(Pdb++) list 2504, 3
2504 ->                  raise UnknownExtra(
2505                       "%s has no such extra feature %r" % (self, ext)
2506                   )
2507           return deps
(Pdb++) self
Unknown 7.4.1 (/private/var/folders/gw/w0clrs515zx9x_55zgtpv4mm0000gp/T/pip-build-zo2eu1/sentry/pip-egg-info/sentry.egg-info)

msabramo · 2015-03-12T19:50:45Z

OK, I think I found a fix for the above issue. I'll file a separate PR for it.

derwolfe · 2015-03-12T19:53:38Z

@msabramo - Great! When that error is fixed, some of my new tests can be updated as well.

msabramo · 2015-03-12T19:55:54Z

Here's the PR I mentioned: #2530

msabramo · 2015-03-12T20:00:10Z

If I merge the change here with the change in #2530:

$ git log -n 1
commit 631a1827806822875862db520deaf2460cd60c5d
Merge: 97b03f4 a12ecdf
Author: Marc Abramowitz <marc@marc-abramowitz.com>
Date:   Thu Mar 12 12:57:57 2015 -0700

    Merge branch 'get_dist_handle_when_egg_info_path_has_trailing_slash' into pr_2142_get_dist

then I get correct behavior:

$ pip install -U 'sentry[lol]'
Collecting sentry[lol]
  Using cached sentry-7.4.1.tar.gz
  sentry 7.4.1 does not provide the extra 'lol'
...

msabramo · 2015-03-12T20:01:18Z

Also fixes #852 ?

Yes. I believe it does.

warn user when requested extra does not exist - 2138

derwolfe changed the title ~~Extras user warn 2138~~ Extras user warn 2138 - WIP Nov 26, 2014

derwolfe force-pushed the extras-user-warn-2138 branch from 509c99c to acf07b6 Compare December 16, 2014 13:13

derwolfe added 3 commits December 30, 2014 17:44

add tests back in

8900f4f

log an error when an extra is requested that doesn't exist

deabed5

logging working a bit better

91f8010

Need to access the extras defined on the distribution itself. This way I could just check for membership in the set of extras defined, and throw an error for each, instead of stopping entirely.

derwolfe force-pushed the extras-user-warn-2138 branch from ade83b9 to 91f8010 Compare December 31, 2014 01:30

derwolfe added 2 commits December 30, 2014 19:41

working on adding tests. It looks like the case of multiple extras no…

fec9f4f

…t exisiting isn't handled

phrasing

53f3346

derwolfe added 8 commits February 21, 2015 17:07

Merge branch 'develop' into extras-user-warn-2138

89bff4a

trying to tease out dist and requires information

b079184

find missing packages first

8266863

naming and return type

db50fe6

fix indentation

6c5edf5

warning showing correctly

c5d331b

fix tests; still need to get the version number

5fcf2d6

tests working, need coverage on two new fucntions

5d3d8ab

derwolfe changed the title ~~Extras user warn 2138 - WIP~~ warn user when requested extra does not exist - 2138 - WIP Feb 24, 2015

derwolfe added 5 commits February 24, 2015 20:30

Cleanup the functional tests, return a tuple instead of list.

2e30c47

shorten line

cb1962e

fix formatting in tests, comments

8a09436

add unit tests for missing and available functions

d3e207e

spacing

19cf33e

derwolfe changed the title ~~warn user when requested extra does not exist - 2138 - WIP~~ warn user when requested extra does not exist - 2138 Feb 26, 2015

xavfernandez reviewed Feb 26, 2015
View reviewed changes

spacing, sort sets prior to pushing into logger, dist.requires

4dfbd2e

sigmavirus24 reviewed Mar 10, 2015
View reviewed changes

make splitting of lines/endings consistent /sigmavirus24

3a1a45a

msabramo reviewed Mar 11, 2015
View reviewed changes

derwolfe added 2 commits March 11, 2015 07:18

nonexistant/nonexistent - msabramo comment

55585d0

fix test names, non_existant/nonexistent

97b03f4

msabramo mentioned this pull request Mar 12, 2015

Make get_dist handle egg_info_path w/ slash at end #2530

Merged

msabramo mentioned this pull request Mar 12, 2015

Installing a package with extras that are invalid should error #852

Closed

dstufft added a commit that referenced this pull request Mar 16, 2015

Merge pull request #2142 from derwolfe/extras-user-warn-2138

cccefe1

warn user when requested extra does not exist - 2138

dstufft merged commit cccefe1 into pypa:develop Mar 16, 2015

dstufft mentioned this pull request Mar 16, 2015

Present a warning if user attempts to install a non-existant extra option #2138

Closed

kmike mentioned this pull request Mar 27, 2017

Issue with running scrapy spider from script. scrapy/scrapy#2473

Closed

jamadden mentioned this pull request Jun 29, 2018

UnknownExtra: html5lib 0.9999999 has no such extra feature 'datrie' buildout/buildout#457

Closed

lock bot added the auto-locked Outdated issues that have been locked by automation label Jun 4, 2019

lock bot locked as resolved and limited conversation to collaborators Jun 4, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

warn user when requested extra does not exist - 2138 #2142

warn user when requested extra does not exist - 2138 #2142

derwolfe commented Nov 26, 2014

derwolfe commented Nov 26, 2014

dstufft commented Nov 26, 2014

derwolfe commented Nov 26, 2014

msabramo commented Dec 16, 2014

derwolfe commented Dec 16, 2014

derwolfe commented Dec 19, 2014

derwolfe commented Dec 20, 2014

derwolfe commented Jan 7, 2015

derwolfe commented Feb 26, 2015

xavfernandez Feb 26, 2015

derwolfe Feb 26, 2015

derwolfe Feb 26, 2015

msabramo Feb 26, 2015

derwolfe Feb 26, 2015

derwolfe commented Feb 26, 2015

sigmavirus24 Mar 10, 2015

msabramo Mar 11, 2015

derwolfe commented Mar 11, 2015

rbu commented Mar 12, 2015

msabramo commented Mar 12, 2015

msabramo commented Mar 12, 2015

msabramo commented Mar 12, 2015

derwolfe commented Mar 12, 2015

msabramo commented Mar 12, 2015

msabramo commented Mar 12, 2015

msabramo commented Mar 12, 2015

warn user when requested extra does not exist - 2138 #2142

warn user when requested extra does not exist - 2138 #2142

Conversation

derwolfe commented Nov 26, 2014

derwolfe commented Nov 26, 2014

dstufft commented Nov 26, 2014

derwolfe commented Nov 26, 2014

msabramo commented Dec 16, 2014

derwolfe commented Dec 16, 2014

derwolfe commented Dec 19, 2014

derwolfe commented Dec 20, 2014

derwolfe commented Jan 7, 2015

derwolfe commented Feb 26, 2015

xavfernandez Feb 26, 2015

Choose a reason for hiding this comment

derwolfe Feb 26, 2015

Choose a reason for hiding this comment

derwolfe Feb 26, 2015

Choose a reason for hiding this comment

msabramo Feb 26, 2015

Choose a reason for hiding this comment

derwolfe Feb 26, 2015

Choose a reason for hiding this comment

derwolfe commented Feb 26, 2015

sigmavirus24 Mar 10, 2015

Choose a reason for hiding this comment

msabramo Mar 11, 2015

Choose a reason for hiding this comment

derwolfe commented Mar 11, 2015

rbu commented Mar 12, 2015

msabramo commented Mar 12, 2015

msabramo commented Mar 12, 2015

msabramo commented Mar 12, 2015

derwolfe commented Mar 12, 2015

msabramo commented Mar 12, 2015

msabramo commented Mar 12, 2015

msabramo commented Mar 12, 2015