"500 Server Error: Internal Server Error" when there is an unintentional trailing newline in the password

[a-metz]

Describe the bug
When uploading distributions with twine I get the error HTTPError: 500 Server Error: Internal Server Error when there is an unintentional trailing newline in the password. This is confusing as I would expect the error 403 Client Error: Invalid or non-existent authentication information. or even an automatic stripping of whitepace.

This happens easily when using continuous deployment and authentication via github secrets, as it is easy to accidentally copy the token from the PyPI Add API token page with a trailing newline. This is then preserved when entering it into a textfield and thus included in the environment variable set by the runner.

As the server seems to be returning a confusing message I added this issue here instead of in the twine repo. Hope it's correct here.

Expected behavior
Receive authentication error or automatic stripping of whitespace

To Reproduce
Note the newline at the end of the token:

$ twine upload --repository-url https://test.pypi.org/legacy/ -u "__token__" -p "pypi-XXX
" dist/*

Uploading distributions to https://test.pypi.org/legacy/
Uploading somepackage-0.1-py3-none-any.whl
100%|██████████████████████████████████████████| 9.74k/9.74k [00:00<00:00, 12.0kB/s]
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 1 of 5
[...]
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 5 of 5
NOTE: Try --verbose to see response content.
HTTPError: 500 Server Error: Internal Server Error for url: https://test.pypi.org/legacy/

Without the newline at the end of the token the behavior is better:

$ twine upload --repository-url https://test.pypi.org/legacy/ -u "__token__" -p "pypi-XXX" dist/*

Uploading distributions to https://test.pypi.org/legacy/
Uploading somepackage-0.1-py3-none-any.whl
100%|██████████████████████████████████████████| 9.74k/9.74k [00:00<00:00, 10.4kB/s]
NOTE: Try --verbose to see response content.
HTTPError: 403 Client Error: Invalid or non-existent authentication information. for url: https://test.pypi.org/legacy/

My Platform

$ python --version
Python 3.6.9
$ twine --version
twine version 3.1.1 (pkginfo: 1.5.0.1, requests: 2.22.0, setuptools: 45.1.0,
requests-toolbelt: 0.9.1, tqdm: 4.41.1)

[di]

Thanks! I think this is the full stacktrace if anyone would like to fix it:

Error: Incorrect padding
  File "raven/middleware.py", line 20, in common_exception_handling
    yield
  File "raven/middleware.py", line 100, in __call__
    iterable = self.application(environ, start_response)
  File "warehouse/utils/wsgi.py", line 99, in __call__
    return self.app(environ, start_response)
  File "warehouse/utils/wsgi.py", line 83, in __call__
    return self.app(environ, start_response)
  File "warehouse/utils/wsgi.py", line 71, in __call__
    return self.app(environ, start_response)
  File "whitenoise/base.py", line 86, in __call__
    return self.application(environ, start_response)
  File "pyramid/router.py", line 270, in __call__
    response = self.execution_policy(environ, self)
  File "pyramid_retry/__init__.py", line 121, in retry_policy
    response = router.invoke_request(request)
  File "pyramid/router.py", line 249, in invoke_request
    response = handle_request(request)
  File "warehouse/sanity.py", line 69, in sanity_tween_ingress
    return handler(request)
  File "warehouse/referrer_policy.py", line 16, in referrer_policy_tween
    response = handler(request)
  File "warehouse/csp.py", line 31, in content_security_policy_tween
    resp = handler(request)
  File "warehouse/config.py", line 82, in require_https_tween
    return handler(request)
  File "pyramid_tm/__init__.py", line 178, in tm_tween
    reraise(*exc_info)
  File "pyramid_tm/compat.py", line 36, in reraise
    raise value
  File "pyramid_tm/__init__.py", line 143, in tm_tween
    response = handler(request)
  File "warehouse/utils/compression.py", line 92, in compression_tween
    response = handler(request)
  File "warehouse/raven.py", line 40, in raven_tween
    return handler(request)
  File "pyramid/tweens.py", line 43, in excview_tween
    response = _error_handler(request, exc)
  File "pyramid/tweens.py", line 17, in _error_handler
    reraise(*exc_info)
  File "pyramid/compat.py", line 179, in reraise
    raise value
  File "pyramid/tweens.py", line 41, in excview_tween
    response = handler(request)
  File "warehouse/cache/http.py", line 74, in conditional_http_tween
    response = handler(request)
  File "warehouse/sanity.py", line 76, in sanity_tween_egress
    return unicode_redirects(handler(request))
  File "pyramid/router.py", line 148, in handle_request
    registry, request, context, context_iface, view_name
  File "pyramid/view.py", line 667, in _call_view
    response = view_callable(context, request)
  File "warehouse/cache/http.py", line 33, in wrapped
    return view(context, request)
  File "warehouse/cache/http.py", line 33, in wrapped
    return view(context, request)
  File "warehouse/csrf.py", line 38, in wrapped
    return view(context, request)
  File "warehouse/metrics/views.py", line 34, in wrapper_view
    return view(context, request)
  File "pyramid/viewderivers.py", line 401, in viewresult_to_response
    result = view(context, request)
  File "pyramid/viewderivers.py", line 144, in _requestonly_view
    response = view(request)
  File "warehouse/forklift/legacy.py", line 760, in file_upload
    if request.authenticated_userid is None:
  File "pyramid/security.py", line 381, in authenticated_userid
    return policy.authenticated_userid(self)
  File "pyramid_multiauth/__init__.py", line 78, in authenticated_userid
    userid = policy.authenticated_userid(request)
  File "pyramid/authentication.py", line 66, in authenticated_userid
    userid = self.unauthenticated_userid(request)
  File "warehouse/macaroons/auth_policy.py", line 89, in unauthenticated_userid
    userid = macaroon_service.find_userid(macaroon)
  File "warehouse/macaroons/services.py", line 79, in find_userid
    m = pymacaroons.Macaroon.deserialize(raw_macaroon)
  File "pymacaroons/macaroon.py", line 47, in deserialize
    return serializer.deserialize(serialized)
  File "pymacaroons/serializers/binary_serializer.py", line 89, in deserialize
    decoded = raw_b64decode(serialized)
  File "pymacaroons/utils.py", line 111, in raw_b64decode
    return raw_urlsafe_b64decode(s)
  File "pymacaroons/utils.py", line 122, in raw_urlsafe_b64decode
    return base64.urlsafe_b64decode(add_base64_padding(s.encode('utf-8')))
  File "python3.7/base64.py", line 133, in urlsafe_b64decode
    return b64decode(s)
  File "python3.7/base64.py", line 87, in b64decode
    return binascii.a2b_base64(s)

(https://sentry.io/organizations/python-software-foundation/issues/1143700661/)

This might be a bug with pymacaroons, we're already attempting to catch a MacaroonDeserializationException:

https://github.com/pypa/warehouse/blob/dfd5de6d1aa91ea591c0bda0c7d20edcab1aad0f/warehouse/macaroons/services.py#L78-L81

[ewdurbin]

It appears that someone has enumerated the various exceptions pymacaroons.Macaroon.deserialize might raise. It'd be great if that were resolved, but we might want to further harden the work from #7424 to handle these other cases?