PipHashing

Status: Released in pip 8. Docs are here.

Secure Mode

If you specify a hash for any requirement in a reqs file (or on the command line?) using a flag (like --sha256), that turns on Secure Mode. Automatic activation means that, in sufficiently up-to-date pips, someone would have to expressly pass --ignore-hashes to be unsafe. (I'm not even sure we should have such an option. It wouldn't be a bad thing to make them edit their reqs file.) In addition, you can turn on Secure Mode by passing --require-hashes, which is useful for bootstrapping a hashless requirements file, since it prints all the hashes of the downloaded files.

In Secure Mode...

• Hashes are required for all requirements. This makes it failsafe; we can always add an option later to relax this if somebody wants to trust their internal server, for example. Hashes in URLs, using the old #md5=... syntax, are considered sufficient if present. The absence of weak hashes in the new flag syntax is a good enough nudge to do the right thing.
• --no-deps is on. Otherwise, untrusted code can come in as dependencies, and the hashes mean nothing. (We may be able to do even better than this: --no-deps just means we throw ImportErrors at runtime if a dependency is unsatisfied. A better behavior would be to notice a missing dependency and complain. In any case, the spirit of this bullet is to make sure nothing unhashed gets installed.)
• Non-== reqs throw an error. Ideally, we would raise this error before installing anything, just to be prompt.
• --egg is probably illegal, because it tells setuptools to go ahead and install dependencies itself, skipping pip's machinery.

Syntax

foo==1.2.3 --sha256=66e7e9fe1dc9b0e84596127c64e320585429310 \
           --sha256=bdb53baa90cf794d54092532b66cae259aa420ed

Note

Someone has rewritten the requirements parser to support multi-line requirements, so there's precedent for this. Jezdez suspects dstufft didn't know about that work when they agreed on the (previous) #sha256 syntax.

Pros

• Can specify hash on the commandline without having to learn anything new. CLI flags have long-proven flexibility for representing a wide range of data., even arbitrary boolean expressions, as in the case of the Unix find command.

  pip install SomePackage[PDF]==3.0 --sha256=abcdef OtherPackage==1.2 --sha256=bcdefa --sha256=cdefab
  

Note

See whether the commandline parser already supports package-associated options like this. If not, it doesn't need to go into the initial release.

• There's room to comment the hashes. People do this in peep all the time: "This one's the wheel." "This is for the Windows binary." "This is the sdist."

Other

• Add an error message if somebody uses an md5 hash. That almost certainly got in there by accident, from pasting a URL from PyPI, and it's not strong enough to give much security.
• Later, we can add freeze-with-requirements functionality to pip. "pip freeze --hash" seems to be consistent with the form of the existing pip freeze options and shorter than "--hashes".
• "pip hash", like "peep hash", should exist. It's easy and useful. It should default to using the best-practice hash of the day: sha256 for now.
• We'll use "standard" hex representations of hashes, not the crazy PEP 427 one from wheels. That way, people can use existing tools like openssl to compute hashes if they want.
• Put "Watch out for setup_requires" in the docs.

Implementation Plan

First release

• Secure Mode as described above
• Support for --sha256 in requirements files
• pip hash

Later

• Support for --sha256 on the commandline. (This is complicated by the introduction of position-significant options. If we want to be consistent, making all arguments position-sensitive, we will break some edge cases, such as in pip install foo --pre bar, in which --pre currently applies to both packages.
• Possible pip freeze additions