include "Python.h" fails in virtual environments managed by uv and conda due to wrong include path

[johannes-mueller]

I set up a project according to the one in the meson-python tutorial. The pip install -e . run works if I run it in a virtual environment created by python -m virtualenv .venv. On environments created by uv or conda it fails with

      [1/2] Compiling C object our_first_module.cpython-312-x86_64-linux-gnu.so.p/our_first_module.c.o
      FAILED: our_first_module.cpython-312-x86_64-linux-gnu.so.p/our_first_module.c.o
      cc -Iour_first_module.cpython-312-x86_64-linux-gnu.so.p -I. -I../.. -I/install/include/python3.12 -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -O3 -fPIC -MD -MQ our_first_module.cpython-312-x86_64-linux-gnu.so.p/our_first_module.c.o -MF our_first_module.cpython-312-x86_64-linux-gnu.so.p/our_first_module.c.o.d -o our_first_module.cpython-312-x86_64-linux-gnu.so.p/our_first_module.c.o -c ../../our_first_module.c
      ../../our_first_module.c:1:10: fatal error: Python.h: No such file or directory
          1 | #include <Python.h>
            |          ^~~~~~~~~~
      compilation terminated.
      ninja: build stopped: subcommand failed.

This is obviously due to a wrong include path -I/install/include/python3.12.

When I use a virtual environment created with virtualenv the ARGS line in build.ninja looks like

ARGS = -Iour_first_module.cpython-311-x86_64-linux-gnu.so.p -I. -I../.. -I/home/<my-user-name>/.miniconda3/include/python3.11 -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -O3 -fPIC

When I use a conda or uv managed environment it looks like

 ARGS = -Iour_first_module.cpython-312-x86_64-linux-gnu.so.p -I. -I../.. -I/install/include/python3.12 -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -O3 -fPIC

Any ideas?

[dnicolodi]

Obviously something is wrong in how conda and uv setup the virtual environment. However, I regularly uses both uv and conda managed virtual environments without any issue. Therefore, the problem is likely local to your installation, or specific to how you use these tools. It is not possible to diagnose the problem further with the provided information.

It would be interesting to check how meson comes to determine the wrong include path. This information is recorded in the verbose meson log in the build directory. Please see here for how to access it when building a Python package

[eli-schwartz]

This looks like a bit of astral-sh/uv#10558 and a bit of astral-sh/uv#9857

Per @zanieb this should be fixed by telling (the latest version of) uv to reinstall python, can you try that?

[johannes-mueller]

Thanks to @dnicolodi and @eli-schwartz.

Yeah, as it turns out is is a broken uv installation. My ~/.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/lib/pkgconfig/python-3.13.pc looks like this

# See: man pkg-config
prefix=/install
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: Python
Description: Build a C extension for Python
Requires:
Version: 3.13
Libs.private: -lpthread -ldl  -lutil
Libs: -L${libdir} 
Cflags: -I${includedir}/python3.13

with the prefix=/install leading to the -I/install

Reinstalling uv from scratch helped.