Frontend image loading fails when set via SHA

[georgysavva]

Hi. We are using the new BuildKit features in our Dockerfile such as RUN --mount=type=cache.
When the docker fronted image is set like this (via tag):

# syntax = docker/dockerfile:1.2.1

Everything works fine.

But if it's set via SHA (exactly the same image with the version 1.2.1) :

# syntax = docker/dockerfile@sha256:6ce32b7af840de740a21ba297f7d1c31e0f1eb0d6b03c0efc03ae0cca9c8fd2a

Docker build fails with this error: failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: exit code: 1

Here is the link to the failing Github action: https://github.com/iotaledger/goshimmer/runs/2130957272
Here is the failing Dockerfile: https://github.com/iotaledger/goshimmer/blob/36f8a67befe32325e60d41720faf3624bf8491e7/Dockerfile
And the complete GH action log:

Run docker build -t iotaledger/goshimmer .
2
  docker build -t iotaledger/goshimmer .
3
  shell: /bin/bash -e {0}
4
  env:
5
    TEST_NAME: autopeering
6
    DOCKER_BUILDKIT: 1
7
#1 [internal] load build definition from Dockerfile
8
#1 sha256:81b95ef4f0c3526e39f64ba88836f200cb33fe5da284fbf20a2f07dc8153effe
9
#1 transferring dockerfile: 1.75kB done
10
#1 DONE 0.0s
11
12

#2 [internal] load .dockerignore

13

#2 sha256:28c59a6999edb3a17da7d9275e4a1763524ffe9255aac2388e3e8993677323db

14

#2 transferring context: 156B done

15

#2 DONE 0.0s

16
17

#4 [auth] docker/dockerfile:pull token for registry-1.docker.io

18

#4 sha256:c5a2cfb6239f601d0c53177c0a1aad05998000f0f893b1268a393320e2adaf75

19

#4 DONE 0.0s

20
21

#3 resolve image config for docker.io/docker/dockerfile@sha256:6ce32b7af840de740a21ba297f7d1c31e0f1eb0d6b03c0efc03ae0cca9c8fd2a

22

#3 sha256:482e5989fd8c3f533ce808aca9b5bf441758a221936245f0a96f7f17366eae5a

23

#3 DONE 0.4s

24
25

#5 docker-image://docker.io/docker/dockerfile@sha256:6ce32b7af840de740a21ba297f7d1c31e0f1eb0d6b03c0efc03ae0cca9c8fd2a

26

#5 sha256:f0937e5cf62fd67a405557ff928aa0617798263d82c353f0c489d5cae075880f

27

#5 resolve docker.io/docker/dockerfile@sha256:6ce32b7af840de740a21ba297f7d1c31e0f1eb0d6b03c0efc03ae0cca9c8fd2a done

28

#5 sha256:c0f387010e18ffd9042a715d26134a644169c9a60ef25c869f2e22d6d5f236bd 4.19MB / 9.68MB 0.1s

29

#5 sha256:6ce32b7af840de740a21ba297f7d1c31e0f1eb0d6b03c0efc03ae0cca9c8fd2a 528B / 528B done

30

#5 sha256:86c932e21798e5a8d71bfc2953075f408c771ba1e111cf08762f5fe4f681358e 1.20kB / 1.20kB done

31

#5 sha256:c0f387010e18ffd9042a715d26134a644169c9a60ef25c869f2e22d6d5f236bd 9.68MB / 9.68MB 0.2s

32

#5 sha256:c0f387010e18ffd9042a715d26134a644169c9a60ef25c869f2e22d6d5f236bd 9.68MB / 9.68MB 0.2s done

33

#5 extracting sha256:c0f387010e18ffd9042a715d26134a644169c9a60ef25c869f2e22d6d5f236bd

34

#5 extracting sha256:c0f387010e18ffd9042a715d26134a644169c9a60ef25c869f2e22d6d5f236bd 0.3s done

35

#5 DONE 0.5s

36

failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: exit code: 1

37

Error: Process completed with exit code 1.

BTW The same Dockerfile that fails on GH actions builds fine on my machine for some reason. Referencing via tag works everywhere.

[tonistiigi]

Seems to work fine for me locally.

@crazy-max Any idea what could be wrong for github? Do they inject some proxy?

[crazy-max]

@tonistiigi

Seems to work fine for me locally.

Me too.

Any idea what could be wrong for github? Do they inject some proxy?

I'm not aware of any kind of proxy on runners. The docker network configuration looks pretty basic to me:

[
    {
        "Name": "bridge",
        "Id": "74c12f96f389b11267707175b9927ea54d14ebab01cb04add8119705758eb2d4",
        "Created": "2021-03-22T17:34:39.631303858Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16",
                    "Gateway": "172.17.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        },
        "Labels": {}
    },
    {
        "Name": "host",
        "Id": "1cf77dc17686220ef8a0cc34907c2c2fa8356e68dbad6da60734f5ef40962ef1",
        "Created": "2021-03-15T03:11:59.356897966Z",
        "Scope": "local",
        "Driver": "host",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": []
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    },
    {
        "Name": "none",
        "Id": "bfb3395b5b1d4dc015c738a9e8ba33812da5ac12b02c71d7a7ea22b6d6547329",
        "Created": "2021-03-15T03:11:59.266816376Z",
        "Scope": "local",
        "Driver": "null",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": []
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]

As well as the runner iptables:

-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A OUTPUT -d 5.34.183.68/32 -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN

5.34.183.68 is odd here but nothing to do with this issue.

And the daemon uses a specific parent group for all containers but don't think that's the issue:

{ "cgroup-parent": "/actions_job" }

[tonistiigi]

Me too.

On local or in github?

[crazy-max]

On local or in github?

Local

[tonistiigi]

This issue appears because docker/dockerfile@sha256:6ce32b7af840de740a21ba297f7d1c31e0f1eb0d6b03c0efc03ae0cca9c8fd2a is a s390x image. If you don't have qemu installed then frontend fails to run. And of course you should never use a platform specific(emulated) image there anyway.

[georgysavva]

Oh, I see. Thanks for helping with this!

@tonistiigi but it's okay to use the platform specific image via SHA that targets linux/amd64 right?
This one: docker/dockerfile@sha256:e3ee2e6b536452d876b1c5aa12db9bca51b8f52b2505178cae6d13e33daeed2b