Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Utilize the mounted disk space for some CI operations #1055

Merged
merged 14 commits into from
Aug 16, 2024
Merged

Conversation

kaxil
Copy link
Member

@kaxil kaxil commented Aug 15, 2024

The CI checkout, build step, & node & node modules take more than >20-30GBs. We have 71GB mounted disk that we can use.

Filesystem      Size  Used Avail Use% Mounted on
  /dev/root        73G   44G   29G  [6](https://github.com/apache/airflow-site/actions/runs/10413031045/job/28839980451#step:8:7)1% /
  tmpfs           7.9G  1[7](https://github.com/apache/airflow-site/actions/runs/10413031045/job/28839980451#step:8:8)2K  7.9G   1% /dev/shm
  tmpfs           3.2G  1.1M  3.2G   1% /run
  tmpfs           5.0M     0  5.0M   0% /run/lock
  /dev/sda15      105M  6.1M   [9](https://github.com/apache/airflow-site/actions/runs/10413031045/job/28839980451#step:8:10)9M   6% /boot/efi
  /dev/sdb1        74G   27G   43G  39% /mnt
  tmpfs           1.6G   12K  1.6G   1% /run/user/[10](https://github.com/apache/airflow-site/actions/runs/10413031045/job/28839980451#step:8:11)01

after docs were built:

df -h
  Filesystem      Size  Used Avail Use% Mounted on
  /dev/root        73G   46G   28G  63% /
  tmpfs           7.9G  172K  7.9G   1% /dev/shm
  tmpfs           3.2G  1.1M  3.2G   1% /run
  tmpfs           5.0M     0  5.0M   0% /run/lock
  /dev/sdb15      105M  6.1M   99M   6% /boot/efi
  /dev/sda1        74G   51G   20G  72% /mnt
  tmpfs           1.6G   12K  1.6G   1% /run/user/1001
  

Once node modules are removed:

Filesystem      Size  Used Avail Use% Mounted on
  /dev/root        73G   45G   28G  62% /
  tmpfs           7.9G  172K  7.9G   1% /dev/shm
  tmpfs           3.2G  1.1M  3.2G   1% /run
  tmpfs           5.0M     0  5.0M   0% /run/lock
  /dev/sda[15](https://github.com/apache/airflow-site/actions/runs/10413031045/job/28839980451#step:22:16)      105M  6.1M   99M   6% /boot/efi
  /dev/sdb1        74G  4.8G   65G   7% /mnt
  tmpfs           1.6G   12K  1.6G   1% /run/user/1001

https://github.com/apache/airflow-site/actions/runs/10413031045/job/28839980451

@kaxil kaxil changed the title Change directory to Mounted dir Utilize the mounted disk space for some CI operations Aug 16, 2024
@ephraimbuddy ephraimbuddy merged commit d72d592 into main Aug 16, 2024
1 check passed
@ephraimbuddy ephraimbuddy deleted the kaxil-patch-1 branch August 16, 2024 01:54
potiuk added a commit to apache/airflow that referenced this pull request Oct 30, 2024
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

Fixes: #41931
potiuk added a commit to apache/airflow that referenced this pull request Oct 30, 2024
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

Fixes: #41931
potiuk added a commit to apache/airflow that referenced this pull request Oct 31, 2024
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

This requires to switch back to public runners for canary builds
as public runners have space available in /mnt

Fixes: #41931
potiuk added a commit to apache/airflow that referenced this pull request Oct 31, 2024
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

This requires to switch back to public runners for canary builds
as public runners have space available in /mnt

Fixes: #41931
potiuk added a commit to apache/airflow that referenced this pull request Oct 31, 2024
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

This requires to switch back to public runners for canary builds
as public runners have space available in /mnt

Fixes: #41931
potiuk added a commit to apache/airflow that referenced this pull request Oct 31, 2024
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

This requires to switch back to public runners for canary builds
as public runners have space available in /mnt

Fixes: #41931
ellisms pushed a commit to ellisms/airflow that referenced this pull request Nov 13, 2024
We are using the same approach as in
apache/airflow-site#1055 where we utilise
/mnt directory to store checked out airflow-site.

This requires to switch back to public runners for canary builds
as public runners have space available in /mnt

Fixes: apache#41931
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants