Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sync v2-9-stable with v2-9-test to release python client v2.9.3 #40930

Merged
merged 5 commits into from
Jul 22, 2024
Merged

Sync v2-9-stable with v2-9-test to release python client v2.9.3 #40930

merged 5 commits into from
Jul 22, 2024

Conversation

utkarsharma2
Copy link
Contributor

@utkarsharma2 utkarsharma2 commented Jul 22, 2024 •
edited
Loading

Cherrypicked missing PRs

@boring-cyborg boring-cyborg bot added area:dev-tools area:production-image Production image improvements and fixes labels Jul 22, 2024
@utkarsharma2 utkarsharma2 changed the base branch from main to v2-9-stable July 22, 2024 10:09
@utkarsharma2
Copy link
Contributor Author

utkarsharma2 commented Jul 22, 2024 •
edited
Loading

Hey @potiuk - On following the release guide, when we running the command to generate the sdist and wheel packages using - breeze release-management prepare-python-client --package-format both --python-client-repo "${CLIENT_REPO_ROOT}" we are observing the deletion on following files in the python client repo, which is unexpected.

deleted:    airflow_client/.gitignore
deleted:    airflow_client/.openapi-generator-ignore
deleted:    airflow_client/.openapi-generator/FILES
deleted:    airflow_client/.openapi-generator/VERSION
deleted:    airflow_client/README.md

Any pointers would be helpful, thanks.

@potiuk
Copy link
Member

potiuk commented Jul 22, 2024

Likely #39181 - I think the breeze changes have not been cherry-picked in 2.9 branch for some time but this one should be easily manually applied.

@utkarsharma2
Copy link
Contributor Author

Likely #39181 - I think the breeze changes have not been cherry-picked in 2.9 branch for some time but this one should be easily manually applied.

Yes, that worked for me. Thanks @potiuk @ephraimbuddy

@potiuk @ephraimbuddy
Only replace "client" subdir when copying sources to python client (#…
9aa6712 
…39181)

When the client is generated, only the "client" subdirectory should
be replaced rather than "airflow_client" parent so that README and
gitignores are not removed.

(cherry picked from commit 57995d0)
@potiuk
Copy link
Member

potiuk commented Jul 22, 2024

BTW. The jobs running here will not work - becuase our self-hosted runners stopped working and we have no way to fix it quickly.

See #40925

I think what you will have to do to fix it is to apply these two that I used to workaround it for now:

Not sure if just cherry-picking will work out-of the box but hopefully yes.

potiuk added 2 commits July 22, 2024 17:35
@potiuk @utkarsharma2
Temporarily switch to public github runners for canary runs (#40913)
bacd043 
We are having problems with self-hosted runners and until we fix
the problems we switch main builds to only run on main with
regular schedule 1x day to serve as a canary build. They will run
longer than on self-hosted runners.

We also add option to trigger the workflow manually from GitHub
interface (by maintainers).

This should cover most tests with the following limitations:

* ARM builds and cache are not going to run (we can still refresh cache
  manually periodicaly to get cache for mac OS Breeze users

* Releasing images via workflow will not work - we can still do
 it manually from preparing workstations

(cherry picked from commit 3a00909)
@potiuk @utkarsharma2
Temporarily comment-out publishing docs in canary runs (#40921)
5c90bc5 
There is not enough space to checkout airflow site on regular
runners so until we fix self-hosted runners, we need to skip
docs publishing.

(cherry picked from commit 173fed0)
@utkarsharma2
Copy link
Contributor Author

BTW. The jobs running here will not work - becuase our self-hosted runners stopped working and we have no way to fix it quickly.

See #40925

I think what you will have to do to fix it is to apply these two that I used to workaround it for now:

* [Temporarily switch to public github runners #40913](https://github.com/apache/airflow/pull/40913)

* [Temporarily comment-out publishing docs in canary runs #40921](https://github.com/apache/airflow/pull/40921)

Not sure if just cherry-picking will work out-of the box but hopefully yes.

Thanks @potiuk I'll try cherry-picking, hopefully should work.

@utkarsharma2
Copy link
Contributor Author

utkarsharma2 commented Jul 22, 2024 •
edited
Loading

@potiuk cerry-picking those changes is causing these test to fail, are there any other changes that I need to cherry-pick?

@potiuk
Copy link
Member

potiuk commented Jul 22, 2024

No. Likely those are tests that have been changed in the meantime. Basically the change was to force ["ubuntu-22.04"] where ["self-hosted", "Linux", "X64"] were before (that was the problem). So simply fixing those tests and replacing "self-hosted" with "ubuntu" will work..

ERROR: The key 'runs-on-as-json-default' has unexpected value:
["ubuntu-22.04"]
Expected value:

["self-hosted", "Linux", "X64"]

@potiuk
Copy link
Member

potiuk commented Jul 22, 2024

But at least the tests are running now.

@utkarsharma2
Copy link
Contributor Author

utkarsharma2 commented Jul 22, 2024 •
edited
Loading

But at least the tests are running now.

@potiuk Yup, they are. They were stuck for a while I had no clue why. Thanks for pointing it out. :)

I have added this commit to rectify the issue.

@utkarsharma2 utkarsharma2 merged commit 70a928c into v2-9-stable Jul 22, 2024
94 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@potiuk potiuk potiuk approved these changes

@ephraimbuddy ephraimbuddy ephraimbuddy approved these changes

@hussein-awala hussein-awala hussein-awala approved these changes

@eladkal eladkal Awaiting requested review from eladkal

@kaxil kaxil Awaiting requested review from kaxil kaxil is a code owner

@jedcunningham jedcunningham Awaiting requested review from jedcunningham jedcunningham is a code owner

Assignees
No one assigned
Labels
area:dev-tools area:production-image Production image improvements and fixes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@utkarsharma2 @potiuk @ephraimbuddy @hussein-awala @jedcunningham