Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor spack-manager to be a plugin #577

Merged
merged 50 commits into from
Dec 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
581aa22
Refactor spack-manager to be a plugin
psakievich Nov 13, 2023
5e22b59
Refactor to starting point for unit-tests
psakievich Nov 14, 2023
099fd21
Add manager module config attributes
psakievich Nov 14, 2023
fbcdde4
Add config test
psakievich Nov 15, 2023
9c73778
Add memory population of manager config
psakievich Nov 15, 2023
d3c845e
Add more testing infrastructure
psakievich Nov 16, 2023
69c0e96
Update machine detection
psakievich Nov 20, 2023
aca9d8b
Update some logic
psakievich Nov 20, 2023
b79f9a0
Refactor detector to be one file per project and not one per machine
psakievich Nov 21, 2023
d3adec5
Add dynamic load of find-machine based on project
psakievich Nov 28, 2023
169f873
Minor structure tweaks
psakievich Nov 28, 2023
24500c7
WIP
psakievich Nov 28, 2023
0233c1c
First create-env test passing
psakievich Nov 28, 2023
5983149
Refactor create_env tests to all pass
psakievich Nov 30, 2023
de9d419
Add branch to action list
psakievich Nov 30, 2023
db69492
Some style, more to come
psakievich Nov 30, 2023
4f1b5c4
Tinker with actions
psakievich Nov 30, 2023
9cd68e4
Split style output
psakievich Nov 30, 2023
63f3791
More style adjustments
psakievich Nov 30, 2023
3306a46
More style
psakievich Nov 30, 2023
22c55ef
Deal with case where project is not defined
psakievich Nov 30, 2023
f36a3bf
Copy python requirements from spack
psakievich Nov 30, 2023
1bf015a
Update tests.yml
psakievich Nov 30, 2023
a3a3110
Update tests.yml
psakievich Nov 30, 2023
e1e548f
Update tests.yml
psakievich Nov 30, 2023
77d3d29
Update tests.yml
psakievich Nov 30, 2023
93d92ab
Update tests.yml
psakievich Nov 30, 2023
14e7c39
Update tests.yml
psakievich Nov 30, 2023
46c585e
Debugging CI failures
psakievich Dec 1, 2023
1053c1d
More CI Debuggery
psakievich Dec 1, 2023
c638c4e
More CI debuggery
psakievich Dec 1, 2023
5a04e65
Debugging linux
psakievich Dec 1, 2023
64d6306
Update imports
psakievich Dec 1, 2023
1785b27
Fix import issues
psakievich Dec 2, 2023
ae5c769
Testing fixes for CI
psakievich Dec 2, 2023
2b4be55
Tweak CI
psakievich Dec 2, 2023
f743af9
Style
psakievich Dec 2, 2023
e1675b9
Style 2.0
psakievich Dec 2, 2023
11b0421
Style 3.0
psakievich Dec 2, 2023
022c10b
Style 4.0
psakievich Dec 2, 2023
0fbbf17
CI Tweaks again
psakievich Dec 2, 2023
6a7e9cc
Remove imports
psakievich Dec 2, 2023
5e05b36
More test tweaks
psakievich Dec 2, 2023
f396d7f
Try passing obj instead of function
psakievich Dec 2, 2023
dd0791a
More test tweaks
psakievich Dec 2, 2023
e674e14
Intermeditate test
psakievich Dec 4, 2023
9370fca
Add a config file to moonlight
psakievich Dec 4, 2023
ad341d1
Style
psakievich Dec 4, 2023
0f4bda5
Style again
psakievich Dec 4, 2023
69251dd
More spack versions
psakievich Dec 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
202 changes: 36 additions & 166 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,200 +5,70 @@ name: Spack-Manager Tests

on:
push:
branches: [ main ]
branches: [ main, develop ]
pull_request:
branches: [ main ]
branches: [ main, develop ]

jobs:
unit-tests:
runs-on: ubuntu-latest
env:
SPACK_MANAGER: ${{github.workspace}}
strategy:
fail-fast: false
matrix:
python-version: [3.7, 3.8, 3.9]
spack-version: [0.21.0, develop]
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Checkout Spack
uses: actions/checkout@v3
with:
repository: spack/spack
ref: ${{ matrix.spack-verions }}
path: spack
- name: Setup Spack ${{ matrix.spack-verion }}
run: echo "$PWD/spack/bin" >> "$GITHUB_PATH"
- name: Install extension
run: |
spack config add config:extensions:[${GITHUB_WORKSPACE}]
spack manager --help
- name: Run unit tests
run: ./scripts/run_tests.sh
run: |
. spack/share/spack/setup-env.sh
spack unit-test --extension manager
style-checks:
runs-on: ubuntu-latest
env:
SPACK_MANAGER: ${{github.workspace}}
strategy:
fail-fast: false
matrix:
python-version: [3.9]
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Spack Style
run: |
# check our spack extension with spack's style guide
source start.sh
spack-start
spack style spack-scripting
spack style scripts
concretization-basic:
runs-on: ${{matrix.os}}
env:
SPACK_MANAGER: ${{github.workspace}}
strategy:
fail-fast: false
matrix:
python-version: [3.7]
os: [ubuntu-latest, macos-latest]
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Run concretization tests
run: |
source start.sh
spack-start
# test and make sure the solver is working for env using clingo
spack solve exawind 1> /dev/null
tests/ci_config_concretization.py -y env-templates/exawind_basic.yaml
tests/ci_config_concretization.py -y env-templates/exawind_matrix.yaml
concretization-matrix:
runs-on: ${{matrix.os}}
env:
SPACK_MANAGER: ${{github.workspace}}
strategy:
fail-fast: false
matrix:
python-version: [3.7]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
- name: Checkout Spack
uses: actions/checkout@v3
with:
submodules: recursive
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Run concretization tests
run: |
source start.sh
spack-start
# test and make sure the solver is working for env using clingo
spack solve exawind 1> /dev/null
tests/ci_config_concretization.py -y env-templates/exawind_matrix.yaml
quick-commands:
runs-on: ubuntu-latest
env:
SPACK_MANAGER: ${{github.workspace}}
strategy:
fail-fast: true
matrix:
python-version: [3.7]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Test quick-commands
run: |
source start.sh
quick-create --name quick-create --spec exawind
spack env deactivate
quick-create-dev --name quick-create-dev --spec amr-wind@main
spack env deactivate
quick-activate ${SPACK_MANAGER}/environments/quick-create-dev
spack env deactivate
quick-create --name spec-with-spaces --spec exawind+hypre build_type=Debug ^amr-wind
reinitialization:
runs-on: ubuntu-latest
env:
SPACK_MANAGER: ${{github.workspace}}
strategy:
fail-fast: true
matrix:
python-version: [3.7]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Test multiple spack-start calls
run: |
source start.sh
spack-start
bash -c "source ${SPACK_MANAGER}/start.sh;spack-start"
snapshot-workflow:
runs-on: ubuntu-latest
env:
SPACK_MANAGER: ${{github.workspace}}
E4S_MACHINE: true
strategy:
fail-fast: true
matrix:
python-version: [3.7]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
repository: spack/spack
path: spack
- name: Setup Spack
run: echo "$PWD/spack/bin" >> "$GITHUB_PATH"
- name: Install dependencies
run: |
ls -lh "$PWD/spack"
ls -lh "$PWD/spack/.github/"
cp ${GITHUB_WORKSPACE}/spack/.github/workflows/style/requirements.txt "$PWD"
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Test snapshot command
run: |
source start.sh
spack-start
spack manager snapshot -m --name git_refs -s exawind+openfast amr-wind+hypre
quick-activate $SPACK_MANAGER/snapshots/exawind/${SPACK_MANAGER_MACHINE}/git_refs
spack env deactivate
spack manager snapshot -m --name regular_versions -r -s exawind+openfast amr-wind+hypre
quick-activate $SPACK_MANAGER/snapshots/exawind/${SPACK_MANAGER_MACHINE}/regular_versions
spack env deactivate



- name: Install extension
run: spack config add config:extensions:[${GITHUB_WORKSPACE}]
- name: Spack Style Manager
run: spack style ${GITHUB_WORKSPACE}/manager
- name: Spack Style Tests
run: spack style ${GITHUB_WORKSPACE}/tests


9 changes: 0 additions & 9 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,16 +1,7 @@
# directories
spack
stage
snapshots
environments
snapshots
golds
logs
tmp
views
modules
cache
configs/user
stage
.bootstrap
.jython_cache
Expand Down
Loading
Loading