Skip to content

Commit

Permalink
ci: Fix broken CI for ASWF 2021 and 2022 containers (AcademySoftwareF…
Browse files Browse the repository at this point in the history
…oundation#4543)

The background here takes some explaining...

The ASWF's prepared Docker containers that reproduce a CentOS7-based
environment corresponding to VFX Platform 2021 and 2022 years contain a
glibc that is too old to run a newer version of "node" -- which GitHub
Actions themselves use. The old one is no longer receiving support, and
GitHub finally changed the policy that was allowing its GHA runners to
still run the old version that worked with those ASWF containers. The
new node version that is happy on the GHA runners requires a glibc newer
than the one in the containers. So that breaks all our CI runs in 2021
and 2022 VFX Platform configurations.

See this thread, if you dare:

https://academysoftwarefdn.slack.com/archives/C0169RX7MMK/p1732574400981949

Anyway, Jean-Francois Panisset proposed and tested a fix, which lets us
continue working (for now) by downloading an old node implemenation
(which is happy with the old glibc) and installing it atop the
conainers, so it's this old one that the GH actions will use. It feels
like it's all held together by chewing gum and duct tape, but it
unbreaks our CI and hopefully it will keep working until all the studios
have switched fully to RHEL/Alma/Rocky 9.x and we no longer care about
testing on CentOS-7 based containers or supporting the VFX Platform
years that are too old to be relevant to the new OS that the studios are
using.

Signed-off-by: Larry Gritz <lg@larrygritz.com>
  • Loading branch information
lgritz committed Nov 29, 2024
1 parent 6cfc439 commit d8f116a
Showing 1 changed file with 155 additions and 23 deletions.
178 changes: 155 additions & 23 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ permissions: read-all

jobs:

aswf:
aswf-old:
name: "VFX${{matrix.vfxyear}} ${{matrix.desc}}"
strategy:
fail-fast: false
Expand Down Expand Up @@ -145,26 +145,6 @@ jobs:
CMAKE_BUILD_TYPE=Debug
CTEST_TEST_TIMEOUT=1200
CTEST_EXCLUSIONS="broken|png-damaged"
- desc: gcc11/C++17 py3.10 boost1.80 exr3.1 ocio2.2
nametag: linux-vfx2023
runner: ubuntu-latest
container: aswftesting/ci-osl:2023-clang15
vfxyear: 2023
cxx_std: 17
python_ver: "3.10"
simd: "avx2,f16c"
fmt_ver: 10.1.1
pybind11_ver: v2.10.0
- desc: gcc11/C++17 py3.11 boost1.82 exr3.2 ocio2.3
nametag: linux-vfx2024
runner: ubuntu-latest
container: aswftesting/ci-osl:2024-clang17
vfxyear: 2024
cxx_std: 17
python_ver: "3.11"
simd: "avx2,f16c"
fmt_ver: 10.1.1
pybind11_ver: v2.10.0
- desc: oldest/hobbled gcc6.3/C++14 py2.7 boost-1.66 exr-2.4 no-sse no-ocio
# Oldest versions of the dependencies that we can muster, and various
# things disabled (no SSE, OCIO, or OpenCV, don't embed plugins).
Expand All @@ -187,6 +167,160 @@ jobs:
USE_OPENCV=0
depcmds: sudo rm -rf /usr/local/include/OpenEXR

runs-on: ${{ matrix.runner }}
container:
image: ${{ matrix.container }}
volumes:
- /node20217:/node20217:rw,rshared
- /node20217:/__e/node20:ro,rshared
env:
CXX: ${{matrix.cxx_compiler}}
CC: ${{matrix.cc_compiler}}
CMAKE_CXX_STANDARD: ${{matrix.cxx_std}}
USE_SIMD: ${{matrix.simd}}
FMT_VERSION: ${{matrix.fmt_ver}}
OPENEXR_VERSION: ${{matrix.openexr_ver}}
PYBIND11_VERSION: ${{matrix.pybind11_ver}}
PYTHON_VERSION: ${{matrix.python_ver}}
ABI_CHECK: ${{matrix.abi_check}}
steps:
# Install nodejs 20 with glibc 2.17, to work around the face that the
# GHA runners are insisting on a node version that is too new for the
# glibc in the ASWF containers prior to 2023.
- name: install nodejs20glibc2.17
if: matrix.old_node == '1'
run: |
curl --silent https://unofficial-builds.nodejs.org/download/release/v20.18.1/node-v20.18.1-linux-x64-glibc-217.tar.xz | tar -xJ --strip-components 1 -C /node20217 -f -
# We would like to use harden-runner, but it flags too many false
# positives, every time we download a dependency. We should use it only
# on CI runs where we are producing artifacts that users might rely on.
# - name: Harden Runner
# uses: step-security/harden-runner@248ae51c2e8cc9622ecf50685c8bf7150c6e8813 # v1.4.3
# with:
# egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs
- uses: actions/checkout@2541b1294d2704b0964813337f33b291d3f8596b # v3.0.2
- name: Prepare ccache timestamp
id: ccache_cache_keys
run: echo "date=`date -u +'%Y-%m-%dT%H:%M:%SZ'`" >> $GITHUB_OUTPUT
- name: ccache
id: ccache
uses: actions/cache@c3f1317a9e7b1ef106c153ac8c0f00fed3ddbc0d # v3.0.4
with:
path: /tmp/ccache
key: ${{github.job}}-${{matrix.nametag}}-${{steps.ccache_cache_keys.outputs.date}}
restore-keys: ${{github.job}}-
- name: Build setup
run: |
${{matrix.setenvs}}
src/build-scripts/ci-startup.bash
- name: Dependencies
run: |
${{matrix.depcmds}}
src/build-scripts/gh-installdeps.bash
- name: Build
run: src/build-scripts/ci-build.bash
- name: Testsuite
if: matrix.skip_tests != '1'
run: src/build-scripts/ci-test.bash
- name: Check out ABI standard
if: matrix.abi_check != ''
uses: actions/checkout@3df4ab11eba7bda6032a0b82a6bb43b11571feac # v4.0.0
with:
ref: ${{matrix.abi_check}}
path: abi_standard
- name: Build ABI standard
if: matrix.abi_check != ''
run: |
mkdir -p abi_standard/build
pushd abi_standard
src/build-scripts/ci-build.bash
popd
- name: Check ABI
if: matrix.abi_check != ''
run: |
src/build-scripts/ci-abicheck.bash ./build abi_standard/build libOpenImageIO libOpenImageIO_Util
- uses: actions/upload-artifact@3cea5372237819ed00197afe530f5a7ea3e805c8 # v3.1.0
if: failure()
with:
name: oiio-${{github.job}}-${{matrix.nametag}}
path: |
build/cmake-save
build/compat_reports
build/testsuite/*/*.*
!build/testsuite/oiio-images
!build/testsuite/openexr-images
!build/testsuite/fits-images
!build/testsuite/j2kp4files_v1_5
aswf:
name: "VFX${{matrix.vfxyear}} ${{matrix.desc}}"
strategy:
fail-fast: false
matrix:
include:
- desc: icc/C++17 py3.10 boost1.76 exr3.1 ocio2.1 qt5.15
nametag: linux-vfx2023-icc
runner: ubuntu-latest
container: aswf/ci-osl:2023-clang15
vfxyear: 2023
cxx_std: 17
python_ver: "3.10"
# simd: "avx2,f16c"
pybind11_ver: v2.10.0
# icc MUST use this older FMT version
pybind11_ver: v2.9.0
setenvs: export USE_ICC=1 USE_OPENVDB=0
OIIO_EXTRA_CPP_ARGS="-fp-model=precise"
# For icc, use fp-model precise to eliminate needless LSB errors
# that make test results differ from other platforms.
- desc: icx/C++17 py3.10 boost1.76 exr3.1 ocio2.1 qt5.15
nametag: linux-vfx2023-icx
runner: ubuntu-latest
container: aswf/ci-osl:2023-clang15
vfxyear: 2023
cc_compiler: icx
cxx_compiler: icpx
cxx_std: 17
python_ver: "3.10"
simd: "avx2,f16c"
pybind11_ver: v2.9.0
setenvs: export USE_OPENVDB=0
- desc: sanitizers
nametag: sanitizer
runner: ubuntu-latest
container: aswf/ci-osl:2023-clang15
vfxyear: 2023
cc_compiler: clang
cxx_compiler: clang++
cxx_std: 17
python_ver: "3.10"
setenvs: export SANITIZE=address,undefined
OIIO_CMAKE_FLAGS="-DSANITIZE=address,undefined -DUSE_PYTHON=0"
CMAKE_BUILD_TYPE=Debug
CTEST_TEST_TIMEOUT=1200
CTEST_EXCLUSIONS="broken|png-damaged"
- desc: gcc11/C++17 py3.10 boost1.80 exr3.1 ocio2.2
nametag: linux-vfx2023
runner: ubuntu-latest
container: aswftesting/ci-osl:2023-clang15
vfxyear: 2023
cxx_std: 17
python_ver: "3.10"
simd: "avx2,f16c"
fmt_ver: 10.1.1
pybind11_ver: v2.10.0
- desc: gcc11/C++17 py3.11 boost1.82 exr3.2 ocio2.3
nametag: linux-vfx2024
runner: ubuntu-latest
container: aswftesting/ci-osl:2024-clang17
vfxyear: 2024
cxx_std: 17
python_ver: "3.11"
simd: "avx2,f16c"
fmt_ver: 10.1.1
pybind11_ver: v2.10.0

# Test ABI stability. `abi_check` is the version or commit that we
# believe is the current standard against which we don't want to
# break the ABI. Basically, we will build that version as well as
Expand Down Expand Up @@ -220,8 +354,6 @@ jobs:
PYBIND11_VERSION: ${{matrix.pybind11_ver}}
PYTHON_VERSION: ${{matrix.python_ver}}
ABI_CHECK: ${{matrix.abi_check}}
ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION: node16
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
steps:
# We would like to use harden-runner, but it flags too many false
# positives, every time we download a dependency. We should use it only
Expand Down

0 comments on commit d8f116a

Please sign in to comment.