Improve handling for hashing unknown packages

[jurre]

Original issue: #7907

Dependabot attempts to resolve package hashes from PyPI, but some packages are hosted on private registries. To address this, we iterate though the index URLs set in dependaboy.yml and pass them to the native helper until a match is found.

[lorengordon]

@jurre Since this change, in our dependabot-script implementation, I've been getting this error. Any idea what this change actually depends on or expects as far as configuration, in order to successfully setup the python updater?

E, [2024-06-04T13:31:42.916676 #7] ERROR -- : Dependabot encountered an error processing <project> : pip: / : undefined method `replaces_base?' for an instance of Hash.
E, [2024-06-04T13:31:42.916718 #7] ERROR -- : [
    "/home/dependabot/dependabot-script/vendor/ruby/3.3.0/gems/dependabot-python-0.259.0/lib/dependabot/python/file_updater.rb:123:in `any?'",
    "/home/dependabot/dependabot-script/vendor/ruby/3.3.0/gems/dependabot-python-0.259.0/lib/dependabot/python/file_updater.rb:123:in `pip_compile_index_urls'",
    "/home/dependabot/dependabot-script/vendor/ruby/3.3.0/gems/dependabot-python-0.259.0/lib/dependabot/python/file_updater.rb:118:in `updated_requirement_based_files'",
    "/home/dependabot/dependabot-script/vendor/ruby/3.3.0/gems/dependabot-python-0.259.0/lib/dependabot/python/file_updater.rb:36:in `updated_dependency_files'",
    "/home/dependabot/dependabot-script/updater.rb:153:in `block (2 levels) in update'",
    "/home/dependabot/dependabot-script/retries.rb:52:in `with_retries'",
    "/home/dependabot/dependabot-script/updater.rb:105:in `block in update'",
    "/home/dependabot/dependabot-script/updater.rb:99:in `each'",
    "/home/dependabot/dependabot-script/updater.rb:99:in `update'",
    "./dependabot.rb:70:in `block (3 levels) in <main>'",
    "./dependabot.rb:67:in `each'",
    "./dependabot.rb:67:in `block (2 levels) in <main>'",
    "./dependabot.rb:58:in `each'",
    "./dependabot.rb:58:in `block in <main>'",
    "./dependabot.rb:34:in `each'",
    "./dependabot.rb:34:in `<main>'"
]

[lorengordon]

Ok, I'm guessing that since #8967, the Array of Hash for credentials is no longer valid, but still used by dependabot-script. Investigating how to setup the credential using the class...

[lorengordon]

Yep, that was it. Fixed it, #8967 (comment)

[ezio-melotti]

@jurre: I think there is a problem with this 4th arg that is causing the issues seen in:

• (python) fatal error upgrading cryptography; calling get_dependency_hash fails #10631

In some cases, that arg ends up being "/pypi/<package-name>/json" (e.g. "/pypi/zope-interface/json") which is not a full URL and causes an error when passed as is to hashin.get_package_hashes in python/helpers/lib/hasher.py.
https://pypi.org/pypi/zope-interface/json is a valid URL, so either the caller should provide the full URL, or some function down the line should concatenate that path to the domain to form a full URL before calling hashin.get_package_hashes.