bug: Test workflow does not fail for unsupported Python versions

[ReubenFrankel]

Singer SDK Version

0.35.0

Is this a regression?

• Yes

Python Version

NA

Bug scope

Other

Operating System

GitHub Actions

Description

When an unsupported Python version is specified in the test workflow matrix, Poetry tries to find a different compatible Python version to install, and will use that instead if it one is available. This produces false-positives for test runs - for example, I expect the Python 3.7 test to fail here: https://github.com/ReubenFrankel/tap-f1/actions/runs/7801705432/job/21277456475#step:5:13

Code

...
The currently activated Python version 3.7.17 is not supported by the project (>=3.8,<4).
Trying to find and use a compatible version. 
Using python3.10 (3.10.12)
Creating virtualenv tap-f1-gIbUqA1n-py3.10 in /home/runner/.cache/pypoetry/virtualenvs
Installing dependencies from lock file
...

[edgarrmondragon]

Thanks for logging @ReubenFrankel. I'm not sure if Poetry has a config that'd help with this, or if the solution is not using Poetry in the cookiecutter's included test workflow...

[ReubenFrankel]

@edgarrmondragon I think this might be possible to do with

    - name: Install dependencies
      run: |
        poetry env use ${{ matrix.python-version }}
        poetry install

$ poetry env use 3.7
The specified Python version (3.7.16) is not supported by the project (>=3.8,<4).
Please choose a compatible version or loosen the python constraint specified in the pyproject.toml file.

[edgarrmondragon]

@edgarrmondragon I think this might be possible to do with

    - name: Install dependencies
      run: |
        poetry env use ${{ matrix.python-version }}
        poetry install

$ poetry env use 3.7
The specified Python version (3.7.16) is not supported by the project (>=3.8,<4).
Please choose a compatible version or loosen the python constraint specified in the pyproject.toml file.

@ReubenFrankel Does that exit with a code 1?

[ReubenFrankel]

Yep - working example: https://github.com/Matatika/tap-msaccess/actions/runs/7851492866/job/21428648454

I added continue-on-error: true as well, to keep running the tests for the other Python versions.

[edgarrmondragon]

Ah, that'd be a fix then!

PS: it's nice to finally be able to singer-sdk in the pytest output 😁:

plugins: durations-1.2.0, singer-sdk-0.35.0