-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Repeatable installs via hashing #3137
Changes from 7 commits
e058486
62ac258
9211d6e
3303be0
1e41f01
11dbb92
0c17248
b0ef6ab
f3f73f1
910b82c
4f67374
14506f8
bf0ff80
c62cd71
09008bf
d477ae6
7a0a97c
0e6058b
6f828c3
52111c1
b95599a
3824d73
be4e315
304c90a
05b7ef9
f35ce75
d541304
76983f3
be6dccb
9e5e34e
4c405a0
dcf39bf
7c5e503
e23f596
925e4b4
622b430
ee9d6fb
3af5ffa
f38fc90
4488047
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,6 +10,7 @@ | |
from __future__ import absolute_import | ||
|
||
from functools import partial | ||
import hashlib | ||
from optparse import OptionGroup, SUPPRESS_HELP, Option | ||
import warnings | ||
|
||
|
@@ -523,6 +524,47 @@ def only_binary(): | |
) | ||
|
||
|
||
def _good_hashes(): | ||
"""Return names of hashlib algorithms at least as strong as sha256.""" | ||
# Remove getattr when 2.6 dies. | ||
algos = set( | ||
getattr(hashlib, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should probably get moved to |
||
'algorithms', | ||
('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512'))) | ||
return algos - set(['md5', 'sha1', 'sha224']) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think this is a good way to actually tell what the strength of these algorithms are. It assumes that there can never be an additional algorithm that isn't as strong as sha256 added to Python. Something like: import hashlib
def _good_hashes():
return set([x for x in hashlib.algorithms if hashlib.new(x).digest_size >= 32]) Ideally without a magical 32 in there, for that matter this probably doesn't need to be a function at all does it? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Unfortunately, digest size is not an authoritative indicator of hash strength. I think we have to go with either a whitelist or a blacklist. I chose a blacklist so people would have the ability to use newer, stronger (or even custom) hashes without waiting for a pip update. Would you rather play it safe and go with a whitelist? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Probably a whitelist then. |
||
|
||
|
||
def _merge_hash(option, opt_str, value, parser): | ||
"""Given a value spelled "algo:digest", append the digest to a list | ||
pointed to in a dict by the algo name.""" | ||
if not parser.values.hashes: | ||
parser.values.hashes = {} | ||
try: | ||
algo, digest = value.split(':', 1) | ||
except ValueError: | ||
parser.error('Arguments to %s must be a hash name ' | ||
'followed by a value, like --hash=sha256:abcde...' % | ||
opt_str) | ||
goods = _good_hashes() | ||
if algo not in goods: | ||
parser.error('Allowed hash algorithms for %s are %s.' % | ||
(opt_str, ', '.join(sorted(goods)))) | ||
parser.values.hashes.setdefault(algo, []).append(digest) | ||
|
||
|
||
hash = partial( | ||
Option, | ||
'-H', '--hash', | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does this deserve a short option like |
||
# Hash values eventually end up in InstallRequirement.hashes due to | ||
# __dict__ copying in process_line(). | ||
dest='hashes', | ||
action='callback', | ||
callback=_merge_hash, | ||
type='string', | ||
help="Verify that the package's archive matches this " | ||
'hash before installing. Example: --hash=sha256:abcdef...') | ||
|
||
|
||
########## | ||
# groups # | ||
########## | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -159,6 +159,14 @@ def __init__(self, *args, **kw): | |
|
||
cmd_opts.add_option(cmdoptions.no_clean()) | ||
|
||
cmd_opts.add_option( | ||
'--require-hashes', | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should this be for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Seems reasonable and not too hard, looking at the code. I did all the operative stuff in RequirementSet, after all. But perhaps it could be a separate PR, later, since I'm about out of time to work on this. Also, nothing supports --hash options on the commandline yet, which is a sad hole in the story. That will need either some serious monkeying around with optparse to support interspersed args and options or else your aforementioned switch to |
||
dest='require_hashes', | ||
action='store_true', | ||
help='Perform a provably repeatable installation by requiring a ' | ||
'hash to check each package against. Implied by the presence ' | ||
'of a --hash option on any individual requirement') | ||
|
||
index_opts = cmdoptions.make_option_group( | ||
cmdoptions.index_group, | ||
self.parser, | ||
|
@@ -266,6 +274,7 @@ def run(self, options, args): | |
pycompile=options.compile, | ||
isolated=options.isolated_mode, | ||
wheel_cache=wheel_cache, | ||
require_hashes=options.require_hashes, | ||
) | ||
|
||
self.populate_requirement_set( | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think 2.7 has this before like, 2.7.9 so probably we can't get rid of it until we get rid of 2.7.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://docs.python.org/3/whatsnew/2.7.html suggests it was added in 2.7.0; it lists things added in 2.7.x separately.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I'm stupid I was confusing it with
guaranteed_hashes
.