error: 'PWIN32_MEMORY_RANGE_ENTRY' has not been declared(Fail to install llama-cpp-python)

[kaipol]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

I am trying to install "llama-cpp-python" in myserver.

Current Behavior

An Exception occurred causing a fail as followed.

Environment and Context

Windows11, CUDA 11.8, CMake 3.27.6, Python 3.11.5, g++ 10.3.0

Failure logs

Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [127 lines of output]
      *** scikit-build-core 0.5.1 using CMake 3.27.6 (wheel)
      *** Configuring CMake...
      2023-09-23 18:10:11,837 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None
      loading initial cache file C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\CMakeInit.txt
      -- Building for: MinGW Makefiles
      -- The C compiler identification is GNU 10.3.0
      -- The CXX compiler identification is GNU 10.3.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: C:/TDM-GCC-64/bin/gcc.exe - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: C:/TDM-GCC-64/bin/c++.exe - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.37.2.windows.2")
      fatal: not a git repository (or any of the parent directories): .git
      fatal: not a git repository (or any of the parent directories): .git
      CMake Warning at vendor/llama.cpp/CMakeLists.txt:125 (message):
        Git repository not found; to enable automatic generation of build info,
        make sure Git is installed and the project is a Git repository.


      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- CMAKE_SYSTEM_PROCESSOR: AMD64
      -- x86 detected
      CMake Warning (dev) at CMakeLists.txt:19 (install):
        Target llama has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION.
      This warning is for project developers.  Use -Wno-dev to suppress it.

      CMake Warning (dev) at CMakeLists.txt:28 (install):
        Target llama has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION.
      This warning is for project developers.  Use -Wno-dev to suppress it.

      -- Configuring done (9.4s)
      -- Generating done (0.2s)
      CMake Warning:
        Manually-specified variables were not used by the project:

          LLAMA_OPENBLAS


      -- Build files have been written to: C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build
      *** Building project with MinGW Makefiles...
      Change Dir: 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'

      Run Build Command(s): D:/Scoop/apps/cmake/3.27.6/bin/cmake.exe -E env VERBOSE=1 C:/TDM-GCC-64/bin/mingw32-make.exe -f Makefile
      D:\Scoop\apps\cmake\3.27.6\bin\cmake.exe -SC:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858 -BC:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build --check-build-system CMakeFiles\Makefile.cmake 0
      D:\Scoop\apps\cmake\3.27.6\bin\cmake.exe -E cmake_progress_start C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\CMakeFiles C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\\CMakeFiles\progress.marks
      C:/TDM-GCC-64/bin/mingw32-make.exe  -f CMakeFiles\Makefile2 all
      mingw32-make.exe[1]: Entering directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      C:/TDM-GCC-64/bin/mingw32-make.exe  -f vendor\llama.cpp\CMakeFiles\ggml.dir\build.make vendor/llama.cpp/CMakeFiles/ggml.dir/depend
      mingw32-make.exe[2]: Entering directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      D:\Scoop\apps\cmake\3.27.6\bin\cmake.exe -E cmake_depends "MinGW Makefiles" C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858 C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp\CMakeFiles\ggml.dir\DependInfo.cmake "--color="
      mingw32-make.exe[2]: Leaving directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      C:/TDM-GCC-64/bin/mingw32-make.exe  -f vendor\llama.cpp\CMakeFiles\ggml.dir\build.make vendor/llama.cpp/CMakeFiles/ggml.dir/build
      mingw32-make.exe[2]: Entering directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      [ 10%] Building C object vendor/llama.cpp/CMakeFiles/ggml.dir/ggml.c.obj
      cd /d C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp && C:\TDM-GCC-64\bin\gcc.exe -DGGML_USE_K_QUANTS -D_CRT_SECURE_NO_WARNINGS -D_XOPEN_SOURCE=600 @CMakeFiles/ggml.dir/includes_C.rsp -O3 -DNDEBUG -std=gnu11 -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Wno-unused-function -mf16c -mfma -mavx -mavx2 -MD -MT vendor/llama.cpp/CMakeFiles/ggml.dir/ggml.c.obj -MF CMakeFiles\ggml.dir\ggml.c.obj.d -o CMakeFiles\ggml.dir\ggml.c.obj -c C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp\ggml.c
      [ 20%] Building C object vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-alloc.c.obj
      cd /d C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp && C:\TDM-GCC-64\bin\gcc.exe -DGGML_USE_K_QUANTS -D_CRT_SECURE_NO_WARNINGS -D_XOPEN_SOURCE=600 @CMakeFiles/ggml.dir/includes_C.rsp -O3 -DNDEBUG -std=gnu11 -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Wno-unused-function -mf16c -mfma -mavx -mavx2 -MD -MT vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-alloc.c.obj -MF CMakeFiles\ggml.dir\ggml-alloc.c.obj.d -o CMakeFiles\ggml.dir\ggml-alloc.c.obj -c C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp\ggml-alloc.c
      [ 30%] Building C object vendor/llama.cpp/CMakeFiles/ggml.dir/k_quants.c.obj
      cd /d C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp && C:\TDM-GCC-64\bin\gcc.exe -DGGML_USE_K_QUANTS -D_CRT_SECURE_NO_WARNINGS -D_XOPEN_SOURCE=600 @CMakeFiles/ggml.dir/includes_C.rsp -O3 -DNDEBUG -std=gnu11 -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Wno-unused-function -mf16c -mfma -mavx -mavx2 -MD -MT vendor/llama.cpp/CMakeFiles/ggml.dir/k_quants.c.obj -MF CMakeFiles\ggml.dir\k_quants.c.obj.d -o CMakeFiles\ggml.dir\k_quants.c.obj -c C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp\k_quants.c
      mingw32-make.exe[2]: Leaving directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      [ 30%] Built target ggml
      C:/TDM-GCC-64/bin/mingw32-make.exe  -f vendor\llama.cpp\CMakeFiles\ggml_static.dir\build.make vendor/llama.cpp/CMakeFiles/ggml_static.dir/depend
      mingw32-make.exe[2]: Entering directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      D:\Scoop\apps\cmake\3.27.6\bin\cmake.exe -E cmake_depends "MinGW Makefiles" C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858 C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp\CMakeFiles\ggml_static.dir\DependInfo.cmake "--color="
      mingw32-make.exe[2]: Leaving directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      C:/TDM-GCC-64/bin/mingw32-make.exe  -f vendor\llama.cpp\CMakeFiles\ggml_static.dir\build.make vendor/llama.cpp/CMakeFiles/ggml_static.dir/build
      mingw32-make.exe[2]: Entering directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      [ 40%] Linking C static library libggml_static.a
      cd /d C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp && D:\Scoop\apps\cmake\3.27.6\bin\cmake.exe -P CMakeFiles\ggml_static.dir\cmake_clean_target.cmake
      cd /d C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp && D:\Scoop\apps\cmake\3.27.6\bin\cmake.exe -E cmake_link_script CMakeFiles\ggml_static.dir\link.txt --verbose=1
      C:\TDM-GCC-64\bin\ar.exe qc libggml_static.a CMakeFiles/ggml.dir/ggml.c.obj "CMakeFiles/ggml.dir/ggml-alloc.c.obj" CMakeFiles/ggml.dir/k_quants.c.obj
      C:\TDM-GCC-64\bin\ranlib.exe libggml_static.a
      mingw32-make.exe[2]: Leaving directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      [ 40%] Built target ggml_static
      C:/TDM-GCC-64/bin/mingw32-make.exe  -f vendor\llama.cpp\CMakeFiles\ggml_shared.dir\build.make vendor/llama.cpp/CMakeFiles/ggml_shared.dir/depend
      mingw32-make.exe[2]: Entering directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      D:\Scoop\apps\cmake\3.27.6\bin\cmake.exe -E cmake_depends "MinGW Makefiles" C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858 C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp\CMakeFiles\ggml_shared.dir\DependInfo.cmake "--color="
      mingw32-make.exe[2]: Leaving directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      C:/TDM-GCC-64/bin/mingw32-make.exe  -f vendor\llama.cpp\CMakeFiles\ggml_shared.dir\build.make vendor/llama.cpp/CMakeFiles/ggml_shared.dir/build
      mingw32-make.exe[2]: Entering directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      [ 50%] Linking C shared library ..\..\bin\libggml_shared.dll
      cd /d C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp && D:\Scoop\apps\cmake\3.27.6\bin\cmake.exe -E cmake_link_script CMakeFiles\ggml_shared.dir\link.txt --verbose=1
      D:\Scoop\apps\cmake\3.27.6\bin\cmake.exe -E rm -f CMakeFiles\ggml_shared.dir/objects.a
      C:\TDM-GCC-64\bin\ar.exe qc CMakeFiles\ggml_shared.dir/objects.a @CMakeFiles\ggml_shared.dir\objects1.rsp
      C:\TDM-GCC-64\bin\gcc.exe -O3 -DNDEBUG -shared -o ..\..\bin\libggml_shared.dll -Wl,--out-implib,libggml_shared.dll.a -Wl,--major-image-version,0,--minor-image-version,0 -Wl,--whole-archive CMakeFiles\ggml_shared.dir/objects.a -Wl,--no-whole-archive @CMakeFiles\ggml_shared.dir\linkLibs.rsp
      mingw32-make.exe[2]: Leaving directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      [ 50%] Built target ggml_shared
      C:/TDM-GCC-64/bin/mingw32-make.exe  -f vendor\llama.cpp\CMakeFiles\llama.dir\build.make vendor/llama.cpp/CMakeFiles/llama.dir/depend
      mingw32-make.exe[2]: Entering directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      D:\Scoop\apps\cmake\3.27.6\bin\cmake.exe -E cmake_depends "MinGW Makefiles" C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858 C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp\CMakeFiles\llama.dir\DependInfo.cmake "--color="
      mingw32-make.exe[2]: Leaving directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      C:/TDM-GCC-64/bin/mingw32-make.exe  -f vendor\llama.cpp\CMakeFiles\llama.dir\build.make vendor/llama.cpp/CMakeFiles/llama.dir/build
      mingw32-make.exe[2]: Entering directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      [ 60%] Building CXX object vendor/llama.cpp/CMakeFiles/llama.dir/llama.cpp.obj
      cd /d C:\Users\kaipol\AppData\Local\Temp\tmp_qvkf9bt\build\vendor\llama.cpp && C:\TDM-GCC-64\bin\c++.exe -DGGML_USE_K_QUANTS -DLLAMA_BUILD -DLLAMA_SHARED -D_CRT_SECURE_NO_WARNINGS -D_XOPEN_SOURCE=600 -Dllama_EXPORTS @CMakeFiles/llama.dir/includes_CXX.rsp -O3 -DNDEBUG -std=gnu++11 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -Wno-format-truncation -Wno-array-bounds -mf16c -mfma -mavx -mavx2 -MD -MT vendor/llama.cpp/CMakeFiles/llama.dir/llama.cpp.obj -MF CMakeFiles\llama.dir\llama.cpp.obj.d -o CMakeFiles\llama.dir\llama.cpp.obj -c C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp\llama.cpp
      C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp\llama.cpp: In constructor 'llama_mmap::llama_mmap(llama_file*, bool, bool)':
      C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp\llama.cpp:676:71: error: 'PWIN32_MEMORY_RANGE_ENTRY' has not been declared
        676 |             BOOL (WINAPI *pPrefetchVirtualMemory) (HANDLE, ULONG_PTR, PWIN32_MEMORY_RANGE_ENTRY, ULONG);
            |                                                                       ^~~~~~~~~~~~~~~~~~~~~~~~~
      C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp\llama.cpp:680:38: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'BOOL (*)(HANDLE, ULONG_PTR, int, ULONG)' {aka 'int (*)(void*, long long unsigned int, int, long unsigned int)'} [-Wcast-function-type]
        680 |             pPrefetchVirtualMemory = reinterpret_cast<decltype(pPrefetchVirtualMemory)> (GetProcAddress(hKernel32, "PrefetchVirtualMemory"));
            |                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp\llama.cpp:684:17: error: 'WIN32_MEMORY_RANGE_ENTRY' was not declared in this scope
        684 |                 WIN32_MEMORY_RANGE_ENTRY range;
            |                 ^~~~~~~~~~~~~~~~~~~~~~~~
      C:\Users\kaipol\AppData\Local\Temp\pip-install-u3xuu5_6\llama-cpp-python_66656a6c3553488e9103e73e82089858\vendor\llama.cpp\llama.cpp:685:17: error: 'range' was not declared in this scope
        685 |                 range.VirtualAddress = addr;
            |                 ^~~~~
      vendor\llama.cpp\CMakeFiles\llama.dir\build.make:75: recipe for target 'vendor/llama.cpp/CMakeFiles/llama.dir/llama.cpp.obj' failed
      mingw32-make.exe[2]: *** [vendor/llama.cpp/CMakeFiles/llama.dir/llama.cpp.obj] Error 1
      mingw32-make.exe[2]: Leaving directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      CMakeFiles\Makefile2:197: recipe for target 'vendor/llama.cpp/CMakeFiles/llama.dir/all' failed
      mingw32-make.exe[1]: *** [vendor/llama.cpp/CMakeFiles/llama.dir/all] Error 2
      mingw32-make.exe[1]: Leaving directory 'C:/Users/kaipol/AppData/Local/Temp/tmp_qvkf9bt/build'
      Makefile:134: recipe for target 'all' failed
      mingw32-make.exe: *** [all] Error 2


      *** CMake build failed
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

Steps to Reproduce

Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.

I used "pip install llama-cpp-python" and "set FORCE_CMAKE=1 && set CMAKE_ARGS=-DLLAMA_CUBLAS=on
pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir -vv",but they both failed and had same error above

[Kreijstal]

I have this error, too under msys2 what is going on?

[Kreijstal]

okay, fix is add

#if defined(_WIN32)
#define _WIN32_WINNT 0x0602 // Targetting Windows 8 for PWIN32_MEMORY_RANGE_ENTRY
#endif

On llama.h, enjoy

---

you know kinda funny that the whole llama.cpp raison d'être is because the guy didn't want to use python enviroment (torch), to simply use a model, so he wrote it with 500 lines of cpp, then, instead of using the whole ecosystem, you made bindings to llama.cpp with python instead of already using the already existing torch ecosystem, I find it funny.

[Kreijstal]

btw that fix means you have to compile on windows 8 or above

[abetlen]

@Kreijstal thanks for the fix. It looks like it was merged into llama.cpp about 2 weeks ago in ggerganov/llama.cpp#4405 so it should be in llama-cpp-python in the latest versions.