Poetry Environment neither automatically detected nor shown in "Select Interpreter" list

[macintacos]

As far as I can tell, Poetry support was added in the version of the Python extension that I have, however I'm unable to get it to detect is automatically by VSCode, and it doesn't appear as an interpreter to select with the appropriate command. I know that I can specify the path for it to get the environment manually, but I feel like that defeats the purpose - I'd really like to get it properly set up in an automated fashion 🙂

Output of poetry env info:

Virtualenv
Python:         3.8.9
Implementation: CPython
Path:           /Users/juliant/Library/Caches/pypoetry/virtualenvs/mince-4ZV0U0mz-py3.8
Valid:          True

System
Platform: darwin
OS:       posix
Python:   /Users/juliant/.pyenv/versions/3.8.9

List of interpreters to choose from:

Environment data

• VS Code version: 1.56.1
• Extension version (available under the Extensions sidebar): v2021.5.829140558
• OS and version: macOS 11.2.3
• Python version (& distribution if applicable, e.g. Anaconda): 3.8.9 (pyenv)
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): Poetry
• Relevant/affected Python packages and their versions: Poetry 1.1.6
• Relevant/affected Python-related VS Code extensions and their verions: N/A? (version given above)
• Value of the python.languageServer setting: Pylance

Expected behaviour

the Python extension automatically finds my Poetry environments and (at the very least) suggests it as an interpreter option

Actual behaviour

The interpreter isn't shown in t he dropdown when I use the "Select Interpreter" command.

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

1. Create a project using Poetry
2. Install dependencies
3. Try to select the environment that Poetry generates
4. Be sad when it's not in the list of interpreters to choose from

[karrtikr]

Thanks for the bug report! We just wanted to quickly acknowledge we received it and we will triage this as soon as we can.

[macintacos]

I've worked around this for the time being by just setting Poetry to create virtual environments in the project. But I believe the point of #8372 was to make it so that this was not required.

[karrtikr]

Yes, this seems unexpected. Can you do the following:

• Add "python.logging.level": "info" to your user settings.
• Provide the logs as mentioned in the issue template that appear when you reload VSCode.

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

[kwmcbride]

I wanted to comment to say that I am having the exact same issue. Setting python.venvPath does nothing. I have tried using both the default directory where poetry stores venvs as well as placing it in the project directory. I cannot type in the path either and have it change. It's as if it does not exist for VSCode.

I would be fine if I could get this to work manually!

[adrianisk]

I'm hitting the same issue, here are the logs that I think show what's happening

Info 2021-05-17 17:28:30: Found: /Users/adrian/Library/Caches/pypoetry/virtualenvs/my-project-cMV7Xc7p-py3.6/bin/python --> /Users/adrian/.pyenv/versions/3.6.10/bin/python3.6
Info 2021-05-17 17:28:30: Found: /Users/adrian/Library/Caches/pypoetry/virtualenvs/my-project-cMV7Xc7p-py3.6/bin/python3 --> /Users/adrian/.pyenv/versions/3.6.10/bin/python3.6
Info 2021-05-17 17:28:30: Found: /Users/adrian/Library/Caches/pypoetry/virtualenvs/my-project-cMV7Xc7p-py3.6/bin/python3.6 --> /Users/adrian/.pyenv/versions/3.6.10/bin/python3.6
Info 2021-05-17 17:28:30: Found: /Users/adrian/.pyenv/versions/3.6.10/bin/python --> /Users/adrian/.pyenv/versions/3.6.10/bin/python3.6
Info 2021-05-17 17:28:30: Found: /Users/adrian/.pyenv/versions/3.6.10/bin/python3 --> /Users/adrian/.pyenv/versions/3.6.10/bin/python3.6
Info 2021-05-17 17:28:30: Found: /Users/adrian/.pyenv/versions/3.6.10/bin/python3.6 --> /Users/adrian/.pyenv/versions/3.6.10/bin/python3.6

I see ~/.pyenv/versions/3.6.10/bin/python in my list of interpreters, but not ~/Library/Caches/pypoetry/virtualenvs/my-project-cMV7Xc7p-py3.6/bin/python. Both are symlinks to the same python binary ~/.pyenv/versions/3.6.10/bin/python3.6, I'm wondering if you're somehow using the target path not the symlink path, and then deduping or using the path as the key in a dictionary?

[adrianisk]

Yep

vscode-python/src/client/pythonEnvironments/common/posixUtils.ts

Lines 118 to 137 in 31c41e0

	const binToLinkMap = new Map<string, string[]>();
	for (const searchDir of searchDirs) {
	const paths = await findPythonBinariesInDir(searchDir);
	for (const filepath of paths) {
	// Ensure that we have a collection of unique global binaries by
	// resolving all symlinks to the target binaries.
	try {
	const resolvedBin = await resolveSymbolicLink(filepath);
	if (binToLinkMap.has(resolvedBin)) {
	binToLinkMap.get(resolvedBin)?.push(filepath);
	} else {
	binToLinkMap.set(resolvedBin, [filepath]);
	}
	traceInfo(`Found: ${filepath} --> ${resolvedBin}`);
	} catch (ex) {
	traceError('Failed to resolve symbolic link: ', ex);
	}
	}
	}

[macintacos]

@karrtikr apologies for the delay. Here's the info-level logs from a fresh session:

https://gist.github.com/macintacos/263ecdf3eb5f1d6495f5353417261e0d

[jleaders]

Is it due to macOS pypoetry sometimes symlinking the python interpreter binary? I notice on Windows that's not the case
On my Windows its not symlinked and works fine
On my macOS it IS symlinked (python3.9) and pypoetry fails to be detected

[jleaders]

@macintacos how do you set pypoetry to create venvs in workspace?

[fuji44]

I also encountered this phenomenon, but it has been resolved.

In my case, it was because I upgraded Python from 3.8 to 3.9 after building the .venv and removed the old version. Maybe it wasn't a good idea to delete it.

To solve this, simply delete the .venv directory, do a poetry install and reopen vscode. Only at first I had to select .venv as the inspector, but after deleting the settings in setting.json, it was automatically detected the next time I reopened it.

Just in case, I would like to write about my environment.

os: Windows 10.0.19042
python: 3.8 -> 3.9.5
vscode: 1.56.2
vscode-python: v2021.5.842923320

.venv is set to be created in the project. virtualenvs.in-project = true.

I don't know if the cause is the same for everyone, but I hope this helps.

[macintacos]

@fuji44 unfortunately, deleting/reinstalling, upgrading Python, etc. has not resolved the situation for me.

@jleaders as @fuji44 mentioned at the end there, you can run poetry config virtualenvs.in-project true to have Poetry create a .venv folder in your local project instead of a cached directory somewhere else on your machine. Then you can just select the interpreter as you would normally.

[fuji44]

@macintacos Thanks for reading.
I'd like to add a little more information, because my writing was not good.

I expect that the cause was that I created .venv with Python 3.8 and then deleted Python 3.8.

As a solution, I ran poetry install on Python 3.9 and recreated the .venv.

[macintacos]

@fuji44 okay - just to clarify, the issue I'm bringing up is not about virtual environments that are being creating in the local project (those work fine), this is strictly about virtual environments that are being created by Poetry in the "default" location, meaning environments that are created when virtualenvs.in-project = false. Hope that clears things up.

[MacLeanLuke]

I'm running the command mentioned above. I am not getting any errors, however the venv folder hasn't been created.

[macintacos]

It's a configuration setting. You have to remove your old environment and then poetry install again after setting the config.

[karrtikr]

@macintacos It seems it's because you're not opted into discovery experiment which has the Poetry related functionality. Can you add the following to your User settings and see if it helps,

"python.experiments.optInto": ["pythonDiscoveryModuleWithoutWatcher"]

If it doesn't please provide a fresh set of logs and provide the output for poetry env list command which we use to discover envs.

[macintacos]

Thanks @karrtikr - I'll try that out, however I wasn't under the impression that this was an "experimental" feature, I thought it was just a new feature added to the extension (everything I was reading didn't mention this option as being required to get Poetry to work).

I'll let you know if that gives me any success though.

[karrtikr]

Yep, we launched this as an experiment feature but all users should now be opted in, not sure why you were not. (there's an opt out setting maybe you have that in your User settings)

[macintacos]

Sorry for the delay @karrtikr, but I just tested that out and it works, thanks! For those who happen upon this, you need to remove your old environment, poetry config virtualenvs.in-project false (if you had this set to true before), poetry install, reload VSCode. and when you next go to "Select Interpreter" and search for "poetry" it should show up.

Closing this out, thanks!

[MacLeanLuke]

Thanks for your work on that! @karrtikr