Use correct Poetry config when collecting Poetry projects

[oranav]

Description:
When collecting Poetry projects for caching, a **/poetry.lock glob is used. However, in order to process the Poetry configuration, the "poetry" command is run from the repo's root directory; this causes Poetry to return an invalid configuration when there is a Poetry project inside an inner directory.

Instead of running a single Poetry command, glob for the same pattern, and configure Poetry for every discovered project.

Related issue:
#446

Check list:

• Mark if documentation changes are required.
• Mark if tests were added or updated to cover the changes.

[oranav]

I still need to test this locally, hence this is a draft.

[oranav]

@IvanZosimov Sorry for the long delay! This is finally ready for review. All tests are passing successfully, and I'm using my own branch as an action in another project.

[oranav]

I fixed the failing main workflow on Windows. Good catch.

That being said, I don't understand why e2e-cache fails with Pipenv and Windows. I haven't changed anything related to it. Can it be failing randomly?

[IvanZosimov]

I fixed the failing main workflow on Windows. Good catch.

That being said, I don't understand why e2e-cache fails with Pipenv and Windows. I haven't changed anything related to it. Can it be failing randomly?

Hi, @oranav 👋 Thank you for this PR and sorry for the delay! The problem with e2e test related to pipenv on Windows is most probably caused by recent pipenv minor tag update, I hope that they will repair it. You also have 3 failing unit-tests in Main workflow, could take a look at them?

[oranav]

Thanks @IvanZosimov!

It seems that I've hit an unrelated bug that happens sporadically, which causes tests to timeout in some cases. Take a look at this action for example, which has failed for the same reason.

I've fixed it in this commit. In short, the cache distributor tests were looking for dependency file with their default glob (**/requirements.txt for example), which made it search for the file in the project's root instead of in the tests data folder. I've changed that, and it reduced the tests time on my machine from ~7s to less than 3 seconds.

After we run the workflows again I expect the main workflow to pass.

[oranav]

Hey @IvanZosimov, can you please take a look?

[oranav]

I'm not sure why main workflow failed with Python 2.7 (which I haven't changed at all); I've seen that it failed in other runs as well, so I suspect it's unrelated to my PR.
I rebased my commits on top of main. Can we please re-run?

[oranav]

@dsame @IvanZosimov Can we please re-run the actions?

[oranav]

Sorry again. The problem was indeed unrelated to my PR, and it seems that #535 fixed the problem (for the curious - there is no Python 2.7 for Ubuntu 22.04, and the workflow previously used ubuntu-latest which got upgraded to 22.04). Unfortunately, #535 wasn't included in my rebase; I now rebased on top of it as well. Running again should result in a working workflow. Can we please re-run one more time?

I ran the workflow in my repo just in case, and it passed successfully:
https://github.com/oranav/setup-python/actions/runs/3412737221/jobs/5678633730

[oranav]

@dsame @IvanZosimov Can we please re-run the workflow?
Sorry for the back and forth :)

[dsame]

@oranav can you please explain the purpose of running poetry env use after the .venv already has been added to the list of cached directories? Is it just to create .venv directory?

i mean this https://github.com/actions/setup-python/pull/447/files#diff-5a0a81991280dd0bbc75f051187afdb8ca7cb621a0bbc95b0d8d5d9a30f143a1R48

[oranav]

@dsame It's actually the same code as before, only in a loop (that iterates over all nested projects).
I believe poetry env use tells Poetry which Python binary to use; since there might not be any cached venv (if it's the first run, or if it has been evicted), then it'll initialize the venv with the correct Python binary.

[MaksimZhukov]

Hello @oranav!
Thank you for the contribution! LGTM!

I think documentation needs to be updated as well. At the moment it says that only virtualenv directory will be cached

[oranav]

Woah, I finally triaged the issue. The old action never ran poetry env use correctly, so the envs.toml file was never created; this in turn made Poetry ignore the venv in case it is broken.
With my fix applied, poetry env use is run correctly, therefore envs.toml is created, but the virtualenv directory already exists (from the cache), so it forces Poetry to use the broken venv.

Either way, we'll want to cache to be rewritten after this PR. I suggest that we'll just bump the Poetry cache distributor primary key, so users will receive a cache miss upon the first run of the new action, and from there everything will work as expected.

I've pushed my changes. Can we please re-run the workflows?

[oranav]

Just to be clear: I've re-run the failing case (before the change, after the change, then once again) and it looks okay now.

[oranav]

Anything still missing from my side or can we proceed to merging this? 😊

[oranav]

@brcrista @dsame @agucova Forgot to tag you in the previous comments, trying again in case there were no notifications. :)

[brcrista]

Looks good @oranav ! I think that's a good fix for the compat issue. Thanks for the contribution!

[invisiblepancake]

src/setup-python.ts
The file sets up the Python environment with caching capabilities.
No code appears suspicious or malicious; it primarily uses core modules and standard libraries.
package.json
The file defines dependencies, scripts, and metadata for the project.
Dependencies and scripts appear standard and necessary for the project.
src/utils.ts
This file includes utility functions for handling system-specific tasks and managing versions.
No code appears suspicious or malicious; it includes standard utility functions.

src/cache.ts

Failed to retrieve the file changes.

[invisiblepancake]

Changes Made:

Added a basedir parameter to set the current working directory for the poetry command.

Potential Issues:
No apparent malicious code. The changes ensure the command runs in the correct directory context.

Summary:

The changes introduced in the functions are aimed at improving the handling of multiple Poetry projects by dynamically identifying and processing their configurations.

No suspicious or malicious code patterns were identified.

[invisiblepancake]

chwers

[invisiblepancake]

README.md

Changes Made:

Updated documentation to explain that for poetry, the action will cache virtualenv directories for each poetry project found.
Potential Issues: No issues identified; the change clarifies the caching behavior for poetry.

tests/cache-restore.test.ts

Changes Made:

Added imports for path and State.
Corrected a typo from saveSatetSpy to saveStateSpy.
Updated test cases to include new paths and hash values.
Enhanced test cases to verify that saveState is called with the correct cache paths.

Potential Issues: No issues identified; the changes improve test coverage and fix typos.

[invisiblepancake]

More tuning. hf =)

[invisiblepancake]

When collecting Poetry projects for caching, a '**/poetry.lock' glob is used. However, in order to process the Poetry configuration, the"poetry" command is run from the repo's root directory; this causes
Poetry to return an invalid configuration when there is a Poetry project
inside an inner directory.

Instead of running a single Poetry command, glob for the same pattern,
and run a Poetry command for every discovered project.

[invisiblepancake]

Simplified the condition check by removing the explicit comparison to true.