From f12c0d447b9f662fd81549f62ba99768383f5c45 Mon Sep 17 00:00:00 2001 From: Huy Do Date: Thu, 21 Nov 2024 13:53:47 -0800 Subject: [PATCH] Login to docker.io (#5957) This is an attempt to mitigate the rate limit issue we are seeing when pulling the base image from docker.io https://docs.docker.com/docker-hub/download-rate-limit/#rate-limit. Here is the summary of my support ticket to GitHub about the issue: 1. This behavior is from Docker Hub itself and isn't something GitHub can directly control. You might consider a paid subscription to Docker Hub which increases the rate limit. You would need to pre-authenticate to Docker Hub on these runners before the job starts as GitHub builds the Docker containers before the first explicit step on the job. 2. Alternatively, we could move the images to https://ghcr.io which does not have such rate limits. 3. GitHub has an agreement with Docker so that the rate limits do not apply for publicly accessible images downloaded through a GitHub-hosted runner but this does not apply to self-hosted runners. This is basically the first approach. We could try it out and see if it helps. The worst thing that could happen is to be rate limited again. This goes together with https://github.com/pytorch-labs/pytorch-gha-infra/pull/544 --- .github/actions/calculate-docker-image/action.yml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.github/actions/calculate-docker-image/action.yml b/.github/actions/calculate-docker-image/action.yml index 34c776cd91..74c88e14d4 100644 --- a/.github/actions/calculate-docker-image/action.yml +++ b/.github/actions/calculate-docker-image/action.yml @@ -157,6 +157,14 @@ runs: retry login "${DOCKER_REGISTRY}" + - name: Login to Docker Hub + shell: bash + # It's ok if this steps fails, it would then be an anonymous user like what we used to have + continue-on-error: true + run: | + set -eux + aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token | jq --raw-output '.SecretString' | jq -r .docker_hub_readonly_token | docker login --username pytorchbot --password-stdin + - name: Build docker image if: ${{ steps.calculate-image.outputs.skip != 'true' && (inputs.always-rebuild || steps.check-image.outputs.rebuild) }} env: