"pip install --upgrade distribute" fails with Python 3

[regebro]

Essentially, since pip uses distribute to do the upgrade, the "import distribute" statements in the pip-provided scripts to gather egg_info and install will end up importing it from the downloaded package instead of from the system packages. But that code is Python 2, and you get syntax errors:

from setuptools.dist import _get_unpatched

File "setuptools\dist.py", line 103

except ValueError, e:
                 ^

SyntaxError: invalid syntax

---

Command python setup.py egg_info failed with error code 1 in C:\Users\root\AppData\Local\Temp\pip\build\distribute

Now, I found solutions for this, but they break everything else (see https://github.com/regebro/pip/commit/4c11aaaed1e10eabfbc9fa60a8a8f0e2df813032 ) so I'm not sure this is fixable without a lot of special casing for Distribute. Seeing as this bug only can appear in Python 3 with packages that have a module called "setuptools" it's a pretty marginal edge case. Perhaps just noting that you can't upgrade Distribute with pip under Python 3 but will have to reinstall it instead is an acceptable solution?

[carljm]

Yes, the import setuptools in the installation subprocess command is pretty critical to pip :-)

I'd certainly be happy with a docs note, at least as a stopgap until/unless someone is motivated enough to try to find a better fix. I've often wished there was a command-line flag to invoke python without the current-directory-on-sys.path behavior.

[dholth]

In the meantime can we avoid uninstalling distribute during an upgrade? Things get messy with --ignore-installed and packages requiring distribute.

[carljm]

I'm not sure how avoiding uninstalling distribute would fix this problem?

[dholth]

With some packages that list distribute as a req, pip tries to upgrade distribute, uninstalls the older one, fails, and leaves a broken virtualenv. Possibly mainly with my wheel patches.

On Aug 29, 2012, at 4:52 PM, Carl Meyer notifications@github.com wrote:

I'm not sure how avoiding uninstalling distribute would fix this problem?

—
Reply to this email directly or view it on GitHub.

[carljm]

Ah, I see. Yeah, a patch to make pip ignore distribute as a
dependency-to-be-upgraded (at least on Py3k) seems like a reasonable
temporary workaround for those cases.

[dholth]

How about a general-purpose --ignore=name (although --ignore is taken as short for --ignore-installed)

[pnasrat]

--exclude or something.

On 30 August 2012 14:44, dholth notifications@github.com wrote:

How about a general-purpose --ignore=name (although --ignore is taken as
short for --ignore-installed)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/650#issuecomment-8159745.

[carljm]

I'm not opposed to the --exclude idea. Should get its own ticket.

[dholth]

Avoid a lot of pain by adding to pip/req.py:

for requirement in to_install:

if requirement.name == "distribute": logger.notify(...); continue

[dholth]

Fix for non-2to3 distribute upgrading problems: dholth@341e71d

[dholth]

How does the fix break everything else?

[pnasrat]

@dholth not sure of the context of your last comment - can you clarify.

[dholth]

Last para of bug description

On Sep 6, 2012, at 2:24 AM, Paul Nasrat notifications@github.com wrote:

@dholth not sure of the context of your last comment - can you clarify.

—
Reply to this email directly or view it on GitHub.

[CharlesB2]

Perhaps just noting that you can't upgrade Distribute with pip under Python 3 but will have to reinstall it instead is an acceptable solution?

Having this issue when running pip3 install --upgrade distribute, how can I do it to upgrade distribute?

[litzomatic]

I'm curious as well to how the attempted fix above "breaks everything else"

Also, an excerpt:

install_args = [
    sys.executable, '-c',
-   "import setuptools;__file__=%r;"\
+   "__file__=%r;"\
    "exec(compile(open(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))" % self.setup_py] +\
    list(global_options) + [
    'install',

Is that import that was removed ever required?

[carljm]

That import is quite important, yes, and removing it is what "breaks everything else". Setuptools monkeypatches the distutils framework when it is imported to provide some additional features, and forcing the import of it before executing setup.py is a core pip feature; it's what allows pip to provide uniform handling of projects that do and don't use setuptools in their setup.py. With that import removed, "pip install" will break for any project that has "from distutils.core import setup" rather than "from setuptools import setup" in its setup.py.

[pfmoore]

At a minimum I would hope that it's possible to do at least some of the following:

• add some comments to the documentation
• allow the user to specify that pip should never upgrade distribute (issue Add mechanism to pin/exclude packages from upgrade #654)
• never attempt to upgrade distribute due to implicit dependency resolution (i.e., the user has to explicitly specify it on the command line)
• raise an error with a better explanation of the issue if the user does attempt to manually upgrade distribute with pip

[jameysharp]

In addition to @pfmoore's list, I'd appreciate if pip freeze excluded distribute from the list of installed packages. That would solve the issues I've run into bringing up new developers' virtualenvs on my projects. I'm encountering this issue in purely Python 2.6/2.7 environments; attempting to upgrade distribute with pip seems like bad news regardless of Python version.

[dholth]

We've had features in either bugs or testing that just skip trying to install distribute or setuptools at all... so pip says "I can't do that Dave" instead of "crash".

[dholth]

the install side is the right place to solve this. Or use grep -v

Jamey Sharp notifications@github.com wrote:

In addition to @pfmoore's list, I'd appreciate if pip freeze excluded distribute from the list of installed packages. That would solve the issues I've run into bringing up new developers' virtualenvs on my projects. I'm encountering this issue in purely Python 2.6/2.7 environments; attempting to upgrade distribute with pip seems like bad news regardless of Python version.

—
Reply to this email directly or view it on GitHub.

[antoneliasson]

As a workaround, is it reasonable to use easy_install instead of pip to upgrade distribute? I just did that and nothing seems to have broken in any obvious way.

[regebro]

Absolutely. I don't know if you keep the possibility to uninstall it from pip, but hey, it's distribute. You need it. :-)

This is not an important bug in any way, it's just annoying, and it keeps cropping up because people run into it and post new bug reports. :-)

[antoneliasson]

It may not be an important bug in pip, but as you say, people keep running into it because they want to upgrade some package that happen to explicitly depend on distribute. That makes it pretty important for the users because it breaks the normal workflow.

[qwcode]

with the release of distribute-0.7.3 (and setuptools>=0.8 which is py2/py3 compatible code), this is no longer an issue.

pip install --upgrade distribute works in py3