Docker *-dev tag's are not targetting the dev stage of Dockerfile

[lodu]

Bug description

the Dockerfile contains a dev stage but this is not the stage that is targeted when building and releasing the *-dev tags (e.g.: master-dev on Docker Hub.

When looking on the image layers from the Superset Docker image with tag master-dev: Layer 29 corresponds to Dockerfile L103 and Layer 30 corresponds to L141 . Anything between those is not present in the *-dev containers (which is the entire dev stage). For example: package libx11-xcb1 does not exist (docker run apache/superset:3.1.0-dev apk -e info libx11-xcb1)

How to reproduce the bug

We first noticed this bug due to geckodriver not working on the *-dev images, so I am using that as an example.

1. Locate executable for geckodriver on a *-dev Docker image: docker run apache/superset:3.1.0-dev which geckodriver
   this does not return a value, but as per docs it should.

Now what we expect:

1. Git clone the repo
2. Build the Docker image while using the dev stage as the target stage: docker build -t superset-dev:latest --target dev . (you can remove a lot of the frontend stuff to increase building speed fyi).
3. Then check the location of geckodriver's executable on that image: docker run superset-dev:latest which geckodriver, this returns /usr/local/bin/geckodriver (as expected).

Screenshots/recordings

Result:

Other packages that should be installed on target dev stage.

Expected results:

Superset version

master / latest-dev

Python version

3.9

Node version

16

Browser

Not applicable

Additional context

Slack discussion. Earlier messages in thread are not that relevant.

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[lodu]

I think the fix is super simple: use this variable.

Not sure how to test this fix I created, if anyone can point me in the right direction? I tested the docker buildx build command locally already which seemed to work.

[giftig]

Just for reference, a related slack discussion: https://apache-superset.slack.com/archives/C015WAZL0KH/p1707628146437669

There's a unit test for this in tests/unit_tests/scripts/docker_build.py, so that'll be the best bet for testing this I think.

[lodu]

[...] docker_build.py, so that'll be the best bet for testing this I think.

Thanks, I fixed that one just now.
I cannot test the actions I think unless I set up the entire Github Actions to publish to a new Docker hub repo.

[lodu]

I just created a PR, when a maintainer approves the workflow for building the dev image (docker-build (dev, linux/amd64)) I can actually check in the container whether the issue is fixed.

[mistercrunch]

Thanks for the PR @lodu , merged!

[mistercrunch]

and let me regenerate the 3.1.0 builds as well

[rusackas]

Not sure if we should close this now since the PR merged, or close it when the builds are complete, but feel free to close when it's appropriate, and thanks for the help here @mistercrunch !

[mistercrunch]

Should be all set, closing!