Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bring back Documentation publishing in Canary runs #41931

Closed
potiuk opened this issue Sep 1, 2024 · 1 comment · Fixed by #43509
Closed

Bring back Documentation publishing in Canary runs #41931

potiuk opened this issue Sep 1, 2024 · 1 comment · Fixed by #43509
Assignees
@potiuk @kaxil
Labels
area:CI Airflow's tests and continious integration area:dev-env CI, pre-commit, pylint and other changes that do not change the behavior of the final code area:self-hosted-runners kind:documentation

Comments

@potiuk
Copy link
Member

potiuk commented Sep 1, 2024 •
edited
Loading

Currently, documentation publishing in Canary runs is disabled. This caused several problems (that will get worse over time):

  • we do not test publishing worklfow - and it might stop working any time and break release process
  • the "sphinx inventory" is not published for airlfow and providers which means that docs building will take more time and will lead to potential errors

airflow/.github/workflows/static-checks-mypy-docs.yml

Line 246 in 7cf54a7

# Temporary disabled it until self-hosted ASF runners will be working again

The reason is that public runners default "workspace" disk is too small to fit checked out documentation while being published - and "self-hosted ASF runners" were unstable even if they had bigger disk.

There are few ways we can address it:

It's been disabled in #40921
@potiuk potiuk converted this from a draft issue Sep 1, 2024
@potiuk potiuk moved this from Backlog to Ready in CI / DEV ENV planned work Sep 1, 2024
@potiuk potiuk added area:dev-env CI, pre-commit, pylint and other changes that do not change the behavior of the final code area:CI Airflow's tests and continious integration labels Sep 5, 2024
@potiuk
Copy link
Member Author

potiuk commented Oct 18, 2024

@kaxil - shall you apply the same fix to the workflow as you did in airlfow-site (trick with mounting another volume) - so that auto-publishing of docs and inventory works ?

kaxil added a commit to astronomer/airflow that referenced this issue Oct 19, 2024
@kaxil
Enable Docs publishing CI step
cc68820 
closes apache#41931
This was referenced Oct 19, 2024
Closed
Closed
@potiuk potiuk mentioned this issue Oct 24, 2024
Open
potiuk added a commit that referenced this issue Oct 30, 2024
@potiuk
Bring back documentation publishing to Canary builds
0a2e676 
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

Fixes: #41931
@potiuk potiuk mentioned this issue Oct 30, 2024
Merged
potiuk added a commit that referenced this issue Oct 30, 2024
@potiuk
Bring back documentation publishing to Canary builds
e20d846 
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

Fixes: #41931
@potiuk potiuk self-assigned this Oct 30, 2024
potiuk added a commit that referenced this issue Oct 31, 2024
@potiuk
Bring back documentation publishing to Canary builds
80074fc 
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

This requires to switch back to public runners for canary builds
as public runners have space available in /mnt

Fixes: #41931
potiuk added a commit that referenced this issue Oct 31, 2024
@potiuk
Bring back documentation publishing to Canary builds
8170ac6 
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

This requires to switch back to public runners for canary builds
as public runners have space available in /mnt

Fixes: #41931
potiuk added a commit that referenced this issue Oct 31, 2024
@potiuk
Bring back documentation publishing to Canary builds
b454a44 
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

This requires to switch back to public runners for canary builds
as public runners have space available in /mnt

Fixes: #41931
@potiuk potiuk closed this as completed in 0531e78 Oct 31, 2024
@github-project-automation github-project-automation bot moved this from In progress to Done in CI / DEV ENV planned work Oct 31, 2024
ellisms pushed a commit to ellisms/airflow that referenced this issue Nov 13, 2024
@potiuk @ellisms
Bring back documentation publishing to Canary builds (apache#43509)
0832c0c 
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

This requires to switch back to public runners for canary builds
as public runners have space available in /mnt

Fixes: apache#41931
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@potiuk potiuk

@kaxil kaxil

Labels
area:CI Airflow's tests and continious integration area:dev-env CI, pre-commit, pylint and other changes that do not change the behavior of the final code area:self-hosted-runners kind:documentation
Projects
CI / DEV ENV planned work
Archived in project
Milestone
No milestone
2 participants
@potiuk @kaxil