Skip to content

Commit

Permalink
gh-36627: CI Linux: Build the default platform in one job
Browse files Browse the repository at this point in the history
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes #1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->
This job is the combination of the "default-pre" job and the first half
of the "default" job (with `make ptest` removed). It takes about 4 hours
and so it fits robustly in the 6h job limit. (Test run:
https://github.com/mkoeppe/sage/actions/runs/6727624213/job/18285698639)

Together with #36616, which deliberately clogs the pipeline with ~50
"standard-pre" jobs, this gives the following behavior when a release
tag is pushed:
- "default" is scheduled
- ~45 "standard-pre" jobs are scheduled (runtime 0.5 to 4 hours)
- ~45 "minimal-pre" jobs (with max-parallel = 20) are scheduled (3 to 4
hours)
- this is calibrated (and can be recalibrated later) so as to keep CI
jobs from PRs waiting until the Docker image built by "default" is
available.

During the clogging time, developers can rebase their PRs to the latest
develop.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes #12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- #12345: short description why this is a dependency
- #34567: ...
-->
- Depends on #36616 (merged here)

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: #36627
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee
  • Loading branch information
Release Manager committed Nov 5, 2023
2 parents 407909d + c1d97ca commit 7354bdb
Showing 1 changed file with 7 additions and 27 deletions.
34 changes: 7 additions & 27 deletions .github/workflows/ci-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,39 +34,20 @@ permissions:

jobs:

# standard-pre for the default platform (used by build.yml etc.)
default-pre:
# standard-pre and standard (without ptest) for the default platform (used by build.yml etc.)
default:
uses: ./.github/workflows/docker.yml
with:
# Build from scratch
docker_targets: "with-system-packages configured with-targets-pre"
docker_targets: "with-system-packages configured with-targets-pre with-targets"
# FIXME: duplicated from env.TARGETS
targets_pre: all-sage-local
tox_system_factors: >-
["ubuntu-focal"]
tox_packages_factors: >-
["standard"]
docker_push_repository: ghcr.io/${{ github.repository }}/

# standard for the default platform (used by build.yml etc.)
default:
if: ${{ success() || failure() }}
needs: [default-pre]
uses: ./.github/workflows/docker.yml
with:
# Build incrementally from previous stage (pre)
incremental: true
free_disk_space: true
from_docker_repository: ghcr.io/${{ github.repository }}/
from_docker_target: "with-targets-pre"
docker_targets: "with-targets with-targets-optional"
# FIXME: duplicated from env.TARGETS
targets: build doc-html
targets_optional: ptest
tox_system_factors: >-
["ubuntu-focal"]
tox_packages_factors: >-
["standard"]
["standard"]
docker_push_repository: ghcr.io/${{ github.repository }}/

# All platforms. This duplicates the default platform, but why not,
Expand Down Expand Up @@ -158,8 +139,6 @@ jobs:

minimal-pre:
if: ${{ success() || failure() }}
# It does not really "need" it.
needs: [standard]
uses: ./.github/workflows/docker.yml
with:
# Build from scratch
Expand All @@ -169,8 +148,9 @@ jobs:
tox_packages_factors: >-
["minimal"]
docker_push_repository: ghcr.io/${{ github.repository }}/
# Reduce from 30 to 25 because it may run in parallel with 'standard-sitepackages' above
max_parallel: 25
# Reduced from 30 because it may run in parallel with 'standard-sitepackages' above.
# Calibrated for clogging the job pipeline until the "default" job has finished.
max_parallel: 20

minimal:
if: ${{ success() || failure() }}
Expand Down

0 comments on commit 7354bdb

Please sign in to comment.