Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incorrect behavior in Python module when the sysconfig paths aren't relative to the install prefix #9284

Closed
FFY00 opened this issue Sep 21, 2021 · 4 comments
Closed

Incorrect behavior in Python module when the sysconfig paths aren't relative to the install prefix #9284

FFY00 opened this issue Sep 21, 2021 · 4 comments

Comments

@FFY00
Copy link
Member

FFY00 commented Sep 21, 2021 •
edited
Loading

Describe the bug
When calculating the install paths in the Python module, if a sysconfig path is not relative to install prefix, Meson will have the wrong path.

To Reproduce
Change the sysconfig paths on the Python interpreter to not be relative to the prefix, and try to install a Python module with meson.

Eg.

$ python -m sysconfig
Platform: "linux-x86_64"
Python version: "3.11"
Current installation scheme: "posix_prefix"

Paths:
	data = "/home/anubis/.virtualenvs/meson-cpython"
	include = "/usr/local/include/python3.11d"
	platinclude = "/usr/local/include/python3.11d"
	platlib = "/opt/custom/lib/python3.11/site-packages"
	platstdlib = "/home/anubis/.virtualenvs/meson-cpython/lib/python3.11"
	purelib = "/opt/custom/lib/python3.11/site-packages"
	scripts = "/home/anubis/.virtualenvs/meson-cpython/bin"
	stdlib = "/usr/local/lib/python3.11"

...

project(
    'pure',
    version: '1.0.0',
)

py_mod = import('python')
py = py_mod.find_installation()

py.install_sources('pure.py')
$ DESTDIR=destdir ninja install
[0/1] Installing files.
Installing /home/anubis/git/mesonpy/tests/packages/pure/pure.py to /home/anubis/git/mesonpy/tests/packages/pure/build/destdir/usr/local/opt/custom/lib/python3.11/site-packages

The file would get installed to /usr/local/opt/custom/lib/python3.11/site-packages, which is incorrect.

Expected behavior
As Meson cannot correctly calculate the correct path, it should error out. We could optionally bypass this for a set of known paths, like /opt, but I am a bit unsure about that.

system parameters

  • Is this a cross build or just a plain native build (for the same computer)? No
  • what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.)
  • what Python version are you using e.g. 3.8.0
  • what meson --version? master
  • what ninja --version if it's a Ninja build
@xclaesse
Copy link
Member

xclaesse commented Sep 21, 2021 •
edited
Loading

Meson master should at least warn when this happens. I think the general consensus was that it's impossible to get it always right, and we should add a Meson built-in option to set the python install path.

FFY00 added a commit to FFY00/meson that referenced this issue Sep 21, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
1d69fec 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system sysconfig scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
FFY00 added a commit to FFY00/meson that referenced this issue Sep 21, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
dccf59b 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system sysconfig scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
@FFY00 FFY00 mentioned this issue Sep 21, 2021
Merged
FFY00 added a commit to FFY00/meson that referenced this issue Sep 22, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
8cee5a7 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
FFY00 added a commit to FFY00/meson that referenced this issue Sep 22, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
33755fd 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
FFY00 added a commit to FFY00/meson that referenced this issue Sep 22, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
e204313 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
FFY00 added a commit to FFY00/meson that referenced this issue Sep 22, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
71ca25b 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
FFY00 added a commit to FFY00/meson that referenced this issue Sep 22, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
563923d 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
FFY00 added a commit to FFY00/meson that referenced this issue Sep 22, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
18aaf59 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
FFY00 added a commit to FFY00/meson that referenced this issue Sep 24, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
e3678b5 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
FFY00 added a commit to FFY00/meson that referenced this issue Sep 29, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
5d433bb 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
FFY00 added a commit to FFY00/meson that referenced this issue Sep 29, 2021
@FFY00
modules: python: better handling of the Python paths for Debian
909fe61 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see mesonbuild#9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
xclaesse pushed a commit that referenced this issue Sep 29, 2021
@FFY00 @xclaesse
modules: python: better handling of the Python paths for Debian
05b5a1e 
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see #9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
@FFY00 FFY00 mentioned this issue Oct 14, 2021
Closed
@eli-schwartz
Copy link
Member

This should be fixed by #9788 which adds the ability to specify -Dpython.install_env=auto, causing Meson to not do prefix relocation but rather to directly install to the sysconfig locations (and prefer a venv if a venv is currently active).

@FFY00
Copy link
Member Author

FFY00 commented Mar 6, 2022

Meson will still generate obviously mangled and incorrect paths by default. Passing that option is just a work around.

@rgommers
Copy link
Contributor

rgommers commented Mar 6, 2022

Meson will still generate obviously mangled and incorrect paths by default. Passing that option is just a work around.

IIRC the plan for that is to make -Dpython.install_env=auto the default (please correct me if I'm wrong @eli-schwartz).

@FFY00 FFY00 mentioned this issue Dec 13, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rgommers @xclaesse @eli-schwartz @FFY00