Skip to content

Commit

Permalink
Fix benchmark script to work with poetry:
Browse files Browse the repository at this point in the history
When initially refactoring to use poetry, the Dockerfile used in
the benchmark task stored the virtual env directory within the
package working directory.

However, the github actions benchmark script maps the runner's working
copy of the ucc directory to the dockerfile run of the benchmark script.
This shadows the /ucc/.venv directory.

This fix switches to store the virtual environment directory outside
the package working directory to avoid this shadowing.
  • Loading branch information
bachase committed Feb 5, 2025
1 parent 85ddbb9 commit 0b800c9
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 9 deletions.
9 changes: 7 additions & 2 deletions .github/workflows/ucc-benchmarks.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# This workflow will install Python dependencies, run benchmarks and then publish teh results to github
# It can be tested locally using https://github.com/nektos/act with the command `act push --container-architecture linux/amd64 -j ucc-benchmarks`
# Follow the instructions https://nektosact.com/usage/index.html#skipping-jobs for setting act = true to skip pushing to github
name: Run Benchmarks

on:
Expand All @@ -15,13 +18,13 @@ jobs:
run-benchmarks:
runs-on: ucc-benchmarks-8-core-U22.04
if: contains(github.event.head_commit.message, '[benchmark chore]') == false

steps:
- name: Checkout Code
uses: actions/checkout@v4
with:
ssh-key: ${{ secrets.DEPLOY_KEY }}

# Build the Docker image
- name: Build Docker image
run: docker build -t ucc-benchmark .
Expand All @@ -32,6 +35,7 @@ jobs:
docker run --rm \
-v "/home/runner/work/ucc/ucc:/ucc" \
ucc-benchmark bash -c "
export POETRY_VIRTUALENVS_IN_PROJECT=false POETRY_VIRTUALENVS_PATH=/venv && \
poetry run ./benchmarks/scripts/run_benchmarks.sh 8 && \
poetry run python ./benchmarks/scripts/plot_avg_benchmarks_over_time.py && \
poetry run python ./benchmarks/scripts/plot_latest_benchmarks.py
Expand All @@ -43,6 +47,7 @@ jobs:
git config --global user.email "actions@github.com"
git config --global user.name "GitHub Actions"
- name: Commit and push results
if: ${{ !github.event.act }} # skip during local actions testing
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
.DS_Store
.vscode
.python-version
.act* # local overrides for act github action tests
.idea/
*.dat
*.hdf5
Expand Down
13 changes: 6 additions & 7 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,20 @@ WORKDIR /ucc
# Copy the entire project into the container
COPY . /ucc

ENV POETRY_NO_INTERACTION=1 \
POETRY_VIRTUALENVS_IN_PROJECT=1 \
# Store the virtual env in /venv to avoid clashing with directory
# mapped as part of benchmark github action
ENV POETRY_NO_INTERACTION=true \
POETRY_VIRTUALENVS_IN_PROJECT=false \
POETRY_VIRTUALENVS_CREATE=true \
POETRY_CACHE_DIR=/tmp/poetry_cache
POETRY_CACHE_DIR=/tmp/poetry_cache \
POETRY_VIRTUALENVS_PATH=/venv

# Install (rarely changing) dependencies only using Poetry to leverage docker caching
RUN --mount=type=cache,target=/tmp/poetry_cache poetry install --no-root

# Install the `ucc` package itself
RUN poetry install

# Ensure the virtual environment is properly activated
ENV VIRTUAL_ENV=/ucc/.venv
ENV PATH="/ucc/.venv/bin:$PATH"

# Show installed package details
RUN poetry run pip show ucc

0 comments on commit 0b800c9

Please sign in to comment.