add guideline for missing image when running breeze

[Lee-W]

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[Taragolis]

No such image Error also could be resolve by run one of this command, both of them also support --python option

breeze ci-image pull

or

breeze ci-image build

[potiuk]

Actually this is not the right solution. The problem is that image should be automatically build if it is missing, there should never be the need to pull it manually.

Likely this might be a side effect of some change where not having an image triggers this problem. It might be result of some recent refactors I've done, but user should never need to pull the image of Breeze manually, user should not even have to know the image name, it's merely an implementation detaill

Was there any special you've done (cleaning docker etc.? @Lee-W - can you come up withe a scenario where this could happen ? I might try to reproduce it, but if you more ore less know how you got there, it could help to fix it.

[Lee-W]

Was there any special you've done (cleaning docker etc.? @Lee-W - can you come up withe a scenario where this could happen ? I might try to reproduce it, but if you more ore less know how you got there, it could help to fix it.

Yes, docker system prune -a or simply run docker rmi ghcr.io/apache/airflow/main/ci/python3.10 and then breeze start-airflow will encounter such error

[potiuk]

Ok. I think (maybe you can check) just adding check_and_rebuild_image_if_needed() when the start command run should solve the problem. It I lis for sure done in shell command and default one. Getting to plane now but should be easy to find and fix :) somewhere in 'developer_commands'

[Lee-W]

Sounds good. Let me check where i can add it

[potiuk]

I found the problem and fixed it as part of #35830

The problem is that the check only checked if the image should be upgraded (not whether it was pulled in the first place). I have also found another problem that prevented the image to be automatically build correctly (version suffix was not set to dev0 and there were conflicting requirements). Closing in favour or my PR :).

Thanks for raising the problem :D

[Lee-W]

Wow, it's a huge PR. thanks for helping out

[potiuk]

Actually, that fix was separated out to a slightly smaller one #35862

I could split it out even further if reviewers think it makes sense (I thin that would inflate the number of PRs artifficiall but if others think the PR is still too big to review, I am happy to split it out.

[potiuk]

The part that fixes the problem you had is here:

https://github.com/apache/airflow/pull/35862/files#diff-6ef5b756e45d03d3723ba4a5c4b0f1417b115fe25560f293351c30290ec67b31R127

    """Starts building attempt. Returns false if we should not continue"""
    result = run_command(
        ["docker", "inspect", ci_image_params.airflow_image_name_with_tag],
        capture_output=True,
        text=True,
        check=False,
    )
    if result.returncode != 0:
        return True
    if ci_image_params.skip_image_upgrade_check:
        return False
    if not ci_image_params.force_build and not ci_image_params.upgrade_to_newer_dependencies:
        if not should_we_run_the_build(build_ci_params=ci_image_params):
            return False
    return True

But it's done together with renaming of e "image_check" property in the object - that's why it's easier to keep it together with the rename.

[Lee-W]

Yep, I'm saying that just to praise the good work, not to request to split the PR. It just might take sometime for me to take a deeper look

[potiuk]

Yep, I'm saying that just to praise the good work, not to request to split the PR. It just might take sometime for me to take a deeper look

I am splitting it anyway to make it easier to review in small chunks. The current one is #35875

[RNHTTR]

I just got the following error...

9.887 ERROR: Cannot install apache-airflow[devel-ci]==2.8.0.dev0 because these package versions have conflicting dependencies.
9.887
9.887 The conflict is caused by:
9.887     apache-airflow[devel-ci] 2.8.0.dev0 depends on WTForms<3.1.0
9.887     The user requested (constraint) wtforms==3.1.1

...

Error response from daemon: No such image: ghcr.io/apache/airflow/main/ci/python3.10:latest

Then I ran docker pull ghcr.io/apache/airflow/main/ci/python3.10:latest before running breeze start-airflow again. breeze start-airflow still gave me that error, but I guess it was able to fall back to the image I just pulled.

Now I'm getting some yarn error :(

[potiuk]

@RNHTTR -> Post on slack the whole output of what you have done. (and whether it was fresh install).

Generally speaking you should always:

a) rebase to latest main first
b) if you get build error when you are running breeze and choosing to rebuid image (or in breeze ci-image build) and get conflicts, you should try wuth --upgrade-to-newer-dependencies and breeze should actually advice you to do it. You should have YELLOW warning and an exact command to run starting with "if you see conflict error" or "if you see strange stacktraces"

I recommend to follow a) and b) - and make sure to read what breeze advice you get.

If you follow it and still got error - please post me the output of the command you run after adding --verbose flag.

Unfirtunately seeing just a piece of the output and even not knowing what command generated it and whether you got and followed the advice printed,does not hep too much.