Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: fix tests and use SLE containers #77

Merged
merged 2 commits into from
Nov 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .codespell.ignore.words
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
aas
enque
versionn
88 changes: 57 additions & 31 deletions .github/workflows/shaptools-ci.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,20 @@
---
name: Package CI
# - this workflow will
# - test python version matrix [2.7, 3.6, 3.7, 3.8, 3.9] for coverage
# - uploads coverage data to codeClimate and the results are linked as a badges in the README
# - test on
# - SLE 12 SP5
# - python 2.7
# - SLE 15 SP5
# - python 3.6
# - deliver the package content to the configured repository
# - submit the new package content to the upstream repository
on: [push, pull_request] # yamllint disable-line rule:truthy
env:
PACKAGE_NAME: python-shaptools
TAR_NAME: shaptools
PACKAGE_NAME: salt-shaptools
jobs:
tab:
name: 'tabspace checking'
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04

# Use the Bash shell regardless whether the GitHub Actions runner is ubuntu-latest, macos-latest, or windows-latest
defaults:
Expand All @@ -28,7 +30,7 @@ jobs:
run: make test-tab
codespell:
name: 'spell checking'
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04

# Use the Bash shell regardless whether the GitHub Actions runner is ubuntu-latest, macos-latest, or windows-latest
defaults:
Expand All @@ -42,6 +44,7 @@ jobs:

- name: Install linting tools
run: |
sudo apt-get update
sudo apt-get install -y git python3 python3-pip
python3 -m pip install codespell

Expand All @@ -50,7 +53,7 @@ jobs:

shellcheck:
name: 'script syntax check'
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04

# Use the Bash shell regardless whether the GitHub Actions runner is ubuntu-latest, macos-latest, or windows-latest
defaults:
Expand All @@ -64,14 +67,15 @@ jobs:

- name: Install linting tools
run: |
sudo apt-get update
sudo apt-get install -y git python3 python3-pip shellcheck

- name: shellcheck
run: make test-shellcheck

yamllint:
name: 'yaml linting'
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04

# Use the Bash shell regardless whether the GitHub Actions runner is ubuntu-latest, macos-latest, or windows-latest
defaults:
Expand All @@ -85,15 +89,16 @@ jobs:

- name: Install linting tools
run: |
sudo apt-get update
sudo apt-get install -y git python3 python3-pip
python3 -m pip install codespell
python3 -m pip install yamllint

- name: yamllint
run: make test-yamllint

jsonlint:
name: 'json linting'
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04

# Use the Bash shell regardless whether the GitHub Actions runner is ubuntu-latest, macos-latest, or windows-latest
defaults:
Expand All @@ -107,6 +112,7 @@ jobs:

- name: Install linting tools
run: |
sudo apt-get update
sudo apt-get install -y git python3 python3-pip
python3 -m pip install jsonlint

Expand All @@ -115,7 +121,7 @@ jobs:

mlc:
name: 'markup link checker'
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04

# Use the Bash shell regardless whether the GitHub Actions runner is ubuntu-latest, macos-latest, or windows-latest
defaults:
Expand All @@ -138,36 +144,55 @@ jobs:
run: make test-mlc

python:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
strategy:
# do not fail if other test fails
fail-fast: false
matrix:
os: [ubuntu-20.04]
python-version: [2.7, 3.6, 3.7, 3.8, 3.9]
container:
- registry.suse.com/suse/sles12sp5:latest # python 2.7
- registry.suse.com/bci/bci-base:15.5 # python 3.6
container:
image: ${{ matrix.container }}
steps:
- name: Auth to SCC and minimal dependencies
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need all this stuff?
I mean, I understand that testing in SLEs is good, but we already do this in OBS (where we run this exact tests as far as I know).
In my opinion it is an overkill, and a hussle, as wee need to maintain secrets now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, I will keep it this way for now.
It is the same CI we currently use for https://github.com/SUSE/salt-shaptools.
Trimming it down can be done another time.

run: |
echo "username=${{ secrets.SCC_USERNAME }}" >/etc/zypp/credentials.d/SCCcredentials
echo "password=${{ secrets.SCC_PASSWORD }}" >>/etc/zypp/credentials.d/SCCcredentials
zypper ref -s
zypper -n in -y tar gzip git
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Test python version [2.7, 3.6, 3.7, 3.8, 3.9] and install dependencies
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest pytest-cov mock
- name: Test with pytest
zypper -n in -y make python
if test -f /usr/bin/python3; then
# minimal salt, python packages and compilers
zypper -n in -y salt python3-pip python3-devel gcc
# use current salt version shipped with SLE 15
git clone --branch=openSUSE/release/3006.0 --depth=50 https://github.com/openSUSE/salt ../salt
# python 3.6 - official requirements from salt (works with python >3.6)
pip install -r ../salt/requirements/pytest.txt
pip install -r tests/requirements.3.6.yaml # pinned pytest-cov
else
zypper -n in -y SUSEConnect
SUSEConnect -p sle-module-adv-systems-management/12/x86_64
# minimal salt, python packages and compilers
zypper -n in -y salt python-pip python-devel gcc gcc-c++
# python 2.7 - latest available versions for old python release
pip install --ignore-installed -r tests/requirements.2.7.yaml
# use current salt version shipped with SLE 12
git clone --branch=openSUSE/release/3000.3 --depth=50 https://github.com/openSUSE/salt ../salt
fi
rm ../salt/tests/conftest.py
- name: execute test script
run: make test-python
- name: Publish code coverage
uses: paambaati/codeclimate-action@v2.7.5
if: ${{ env.CC_TEST_REPORTER_ID != null && github.event_name != 'pull_request' }}
env:
CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
with:
coverageLocations: coverage.xml:coverage.py


delivery:
needs: [tab, codespell, shellcheck, yamllint, jsonlint, mlc, python]
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
if: ${{ github.event_name != 'pull_request' }}
container:
image: shap/continuous_deliver
Expand All @@ -192,9 +217,10 @@ jobs:
sed -i 's~%%REPOSITORY%%~${{ github.repository }}~' _service && \
/scripts/upload.sh


submit:
needs: [tab, codespell, shellcheck, yamllint, jsonlint, mlc, python, delivery]
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
if: ${{ github.event_name != 'pull_request' && github.ref == 'refs/heads/master' }}
container:
image: shap/continuous_deliver
Expand Down
2 changes: 1 addition & 1 deletion shaptools/saputils.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

:since: 2020-03-12
"""
#TODO: Add support for other SAPCAR functionalties apart from extraction
#TODO: Add support for other SAPCAR functionalities apart from extraction

import os

Expand Down
11 changes: 11 additions & 0 deletions tests/requirements.2.7.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
setuptools==44.1.1
wheel==0.37.1
pytest-cov==2.3.1
jinja2==2.6
pyyaml==3.12
markupsafe==0.23
requests==2.27
certifi==2020.4.5.1
pyzmq==18.0.1
mock==3.0.0
funcsigs==1.0.2
1 change: 1 addition & 0 deletions tests/requirements.3.6.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pytest-cov==4.0.0