Failure to re-create venv after Poetry update in 5.0.6-0 rpm #2788 #2789
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…#2788 ## Includes - Remove now redundant Py3.6 work-around re Poetry and LANG env var. - Establish LANG from install. - Record PATH & `poetry --version` in poetry-install.txt before venv install.
Dev notes:Test-dev system as per PR: #2787 And so on this system we do have a |
export PIPX_HOME=/opt/pipx
export PIPX_BIN_DIR=/usr/local/bin
export PIPX_MAN_DIR=/usr/local/share/man
printenv PATH
/sbin:/usr/sbin:/usr/local/sbin:/root/.local/bin:/root/bin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin
pipx ensurepath
/usr/local/bin is already in PATH.
⚠ All pipx binary directories have been added to PATH. If you are sure you want to proceed, try again with the '--force' flag.
Otherwise pipx is ready to go! ✨ 🌟 ✨
printenv PATH
/sbin:/usr/sbin:/usr/local/sbin:/root/.local/bin:/root/bin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbinWe have, from our zypper installed pipx-python311 from: https://pipx.pypa.io/stable/docs/
|
|
From a regular root terminal if we adopt the reported PATH: PATH="/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
printenv PATH
/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
export PIPX_HOME=/opt/pipx
export PIPX_BIN_DIR=/usr/local/bin
export PIPX_MAN_DIR=/usr/local/share/man
/usr/local/bin/poetry install --no-interaction --no-ansi >> poetry-install.txt 2>&1results in a successful venv re-creation. |
python3.11 -m pipx install poetry==1.7.1
'poetry' already seems to be installed. Not modifying existing installation in '/opt/pipx/venvs/poetry'. Pass '--force' to force installation.with ls -la /opt/pipx/venvs/poetry/
total 12
drwxr-xr-x 1 root root 92 Jan 25 16:13 .
drwxr-xr-x 1 root root 12 Jan 25 16:12 ..
drwxr-xr-x 1 root root 344 Jan 25 16:13 bin
drwxr-xr-x 1 root root 20 Jan 25 16:12 include
drwxr-xr-x 1 root root 20 Jan 25 16:12 lib
lrwxrwxrwx 1 root root 3 Jan 25 16:12 lib64 -> lib
-rw-r--r-- 1 root root 2564 Jan 25 16:13 pipx_metadata.json
-rw-r--r-- 1 root root 178 Jan 25 16:12 pyvenv.cfgand our pipx installed poetry symlink: which poetry
/usr/local/bin/poetrypoints to: ls -la /usr/local/bin/poetry
lrwxrwxrwx 1 root root 33 Jan 25 16:13 /usr/local/bin/poetry -> /opt/pipx/venvs/poetry/bin/poetryWhich is a venv dir of sorts: But all this works as intended from a root shell env, with the reported rpm %posttrans PATH enforced there-in. |
…ued rockstor#2788 - alternate python version declaration - poetry install via pipx. - diagnostic `pipx list` addition.
and our failure report from the attempted venv re-creation: |
…ued2 rockstor#2788 - Dump env output to aid diagnostic. - Enable full poetry debug logging.
|
Using the new debug log and full env dump we have the following: installer:~ # cat /opt/rockstor/poetry-install.txt
LC_ALL=C
DJANGO_SETTINGS_MODULE=settings
LANG=en_GB.UTF-8
TZ=Europe/Lisbon
SYSTEMD_EXEC_PID=13446
OLDPWD=/
INVOCATION_ID=f722f967dc4a4e2bbaff3f99340a4aad
BOOST_TEST_CATCH_SYSTEM_ERRORS=no
RPM_IgnoreFailedSymlinks=1
VIRTUAL_ENV=/opt/rockstor/.venv
PWD=/opt/rockstor
PIPX_HOME=/opt/pipx
HOME=/root
JOURNAL_STREAM=8:18947
SUPERVISOR_PROCESS_NAME=gunicorn
PIPX_MAN_DIR=/usr/local/share/man
SUPERVISOR_ENABLED=1
ZYPP_IS_RUNNING=14401
PIPX_BIN_DIR=/usr/local/bin
PLAT=linux-x86_64
SHLVL=3
SUPERVISOR_GROUP_NAME=gunicorn
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SUPERVISOR_SERVER_URL=unix:///var/run/supervisor.sock
SERVER_SOFTWARE=gunicorn/19.10.0
_=/usr/bin/env
Poetry (version 1.7.1)
Loading configuration file /opt/rockstor/poetry.toml
Stack trace:
2 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/utils/env/base_env.py:350 in _run
348│ output = ""
349│ else:
→ 350│ output = subprocess.check_output(
351│ cmd, stderr=stderr, env=env, text=True, **kwargs
352│ )
1 /usr/lib64/python3.11/subprocess.py:466 in check_output
464│ kwargs['input'] = empty
465│
→ 466│ return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
467│ **kwargs).stdout
468│
CalledProcessError
Command '['python', '-I', '-W', 'ignore', '-c', 'import sys\n\nif hasattr(sys, "real_prefix"):\n print(sys.real_prefix)\nelif hasattr(sys, "base_prefix"):\n print(sys.base_prefix)\nelse:\n print(sys.prefix)\n']' returned non-zero exit status 2.
at /usr/lib64/python3.11/subprocess.py:571 in run
567│ # We don't call process.wait() as .__exit__ does that for us.
568│ raise
569│ retcode = process.poll()
570│ if check and retcode:
→ 571│ raise CalledProcessError(retcode, process.args,
572│ output=stdout, stderr=stderr)
573│ return CompletedProcess(process.args, retcode, stdout, stderr)
574│
575│
The following error occurred when trying to handle this error:
Stack trace:
14 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/application.py:327 in run
325│
326│ try:
→ 327│ exit_code = self._run(io)
328│ except BrokenPipeError:
329│ # If we are piped to another process, it may close early and send a
13 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/console/application.py:190 in _run
188│ self._load_plugins(io)
189│
→ 190│ exit_code: int = super()._run(io)
191│ return exit_code
192│
12 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/application.py:431 in _run
429│ io.input.interactive(interactive)
430│
→ 431│ exit_code = self._run_command(command, io)
432│ self._running_command = None
433│
11 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/application.py:473 in _run_command
471│
472│ if error is not None:
→ 473│ raise error
474│
475│ return terminate_event.exit_code
10 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/application.py:454 in _run_command
452│
453│ try:
→ 454│ self._event_dispatcher.dispatch(command_event, COMMAND)
455│
456│ if command_event.command_should_run():
9 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/events/event_dispatcher.py:26 in dispatch
24│
25│ if listeners:
→ 26│ self._do_dispatch(listeners, event_name, event)
27│
28│ return event
8 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/cleo/events/event_dispatcher.py:85 in _do_dispatch
83│ break
84│
→ 85│ listener(event, event_name, self)
86│
87│ def _sort_listeners(self, event_name: str) -> None:
7 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/console/application.py:299 in configure_env
297│
298│ env_manager = EnvManager(poetry, io=io)
→ 299│ env = env_manager.create_venv()
300│
301│ if env.is_venv() and io.is_verbose():
6 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/utils/env/env_manager.py:436 in create_venv
434│
435│ cwd = self._poetry.file.path.parent
→ 436│ env = self.get(reload=True)
437│
438│ if not env.is_sane():
5 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/utils/env/env_manager.py:291 in get
289│ base_prefix = self.get_base_prefix()
290│
→ 291│ return VirtualEnv(prefix, base_prefix)
292│
293│ def list(self, name: str | None = None) -> list[VirtualEnv]:
4 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/utils/env/virtual_env.py:45 in __init__
43│ # from inside the virtualenv.
44│ if base is None:
→ 45│ output = self.run_python_script(GET_BASE_PREFIX)
46│ self._base = Path(output.strip())
47│
3 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/utils/env/base_env.py:325 in run_python_script
323│
324│ def run_python_script(self, content: str, **kwargs: Any) -> str:
→ 325│ return self.run(
326│ self._executable,
327│ "-I",
2 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/utils/env/base_env.py:317 in run
315│ def run(self, bin: str, *args: str, **kwargs: Any) -> str:
316│ cmd = self.get_command_from_bin(bin) + list(args)
→ 317│ return self._run(cmd, **kwargs)
318│
319│ def run_pip(self, *args: str, **kwargs: Any) -> str:
1 /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/utils/env/virtual_env.py:99 in _run
97│ def _run(self, cmd: list[str], **kwargs: Any) -> str:
98│ kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
→ 99│ return super()._run(cmd, **kwargs)
100│
101│ def get_temp_environ(
EnvCommandError
Command ['python', '-I', '-W', 'ignore', '-c', 'import sys\n\nif hasattr(sys, "real_prefix"):\n print(sys.real_prefix)\nelif hasattr(sys, "base_prefix"):\n print(sys.base_prefix)\nelse:\n print(sys.prefix)\n'] errored with the following return code 2
Error output:
Unknown option: -I
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Try `python -h' for more information.
at /opt/pipx/venvs/poetry/lib64/python3.11/site-packages/poetry/utils/env/base_env.py:354 in _run
350│ output = subprocess.check_output(
351│ cmd, stderr=stderr, env=env, text=True, **kwargs
352│ )
353│ except CalledProcessError as e:
→ 354│ raise EnvCommandError(e)
355│
356│ return output
357│
358│ def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
|
|
We have a reproducer of sorts. On the target test system (Leap 15.4) we can reproduce our python from a root terminal thus: And from Which looks like newer (Py3.7+ only) Poetry's investigation of the underlying python environment. But But if we specifically invoke the correct python via executable name, as we have requested pipx does for us !!! On a 15.5 system we have no 'python': But again our pipx is supposed to be presenting Poetry with it's python -> python3.11 via it's venv. And the code in the failing command string (-c) is: import sys
if hasattr(sys, "real_prefix"):
print(sys.real_prefix)
elif hasattr(sys, "base_prefix"):
print(sys.base_prefix)
else:
print(sys.prefix)But our issue is the inadvertent invokation by Poetyr of the wrong python version, when pipx is meant to resolve this. |
|
A somewhat incidental run of that code: installer:~ # python -I -W ignore < test.py
Unknown option: -I
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Try `python -h' for more information.
installer:~ # python3.11 -I -W ignore < test.py
/usr |
|
We may have the remains of an in-memory (from deleted venv) pointer to our old python remaining. And we have the following potentially related fix regarding un-setting an environmental variable (
in our update environment, and even thought it references a now deleted .venv, there is evidence of dangling python references here. |
…ued3 rockstor#2788 - unset pre 5.0.3-0 VIRTUAL_ENV environmental variable.
|
Working Web-UI update on base OS of 15.4 (last available installer) from last stable of 4.1.0-0 - to testing 5.0.6-2788:
And post Web-UI (systemd services) resumption we have: installer:~ # pass
Password Store
└── python-keyring
└── rockstor
├── CLIENT_SECRET
├── SECRET_KEY_FALLBACK
└── SECRET_KEY |
|
I will shortly squash this development branch and re-present as a non draft pull request. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Includes
poetry --versionin poetry-install.txt before venv install.