wxPython: 4.1.0 -> 4.1.1 #135607

davidak · 2021-08-24T22:43:25Z

Motivation for this change

This is fixing wxWidgets/Phoenix#1956 (which affects the Timeline package)

Things done

tfmoraes · 2021-08-24T22:54:48Z

@davidak the only problem with wxPython 4.1.1 is that it doesn't compile with wxWidgets from the system. wxPython only compiles if used the wxWidgets included in the wxPython source code. To do that, you just need to remove --use_syswx from buildPhase:

  buildPhase = ''
    ${python.interpreter} build.py -v dox etg --nodoc build_py
  '';

davidak · 2021-08-25T00:12:59Z

@tfmoraes yes, that fixed the problem. upstream issue wxWidgets/Phoenix#1909

now i get this error:

[davidak@gaming:~/code/nixpkgs]$ nix run -f ~/code/nixpkgs/ python3Packages.wxPython_4_1
builder for '/nix/store/h1q5d9jj2x1yghbc0d4i04yva25gyqc8-python3.9-wxPython-4.1.1.drv' failed with exit code 127; last 10 log lines:
  python-config                            : /nix/store/nki9ywqzbvz68vr75kn2r7g1q84f5agy-python3-3.9.6/bin/python3.9-config 
  Asking python-config for pyext '--cflags --libs --ldflags' flags : 00:05:56 runner ['/nix/store/nki9ywqzbvz68vr75kn2r7g1q84f5agy-python3-3.9.6/bin/python3.9-config', '--cflags', '--libs', '--ldflags']
  yes 
  Testing pyext configuration                                      : 00:05:56 runner ['g++', '-fPIC', '-g', '-fwrapv', '-O3', '-I/nix/store/nki9ywqzbvz68vr75kn2r7g1q84f5agy-python3-3.9.6/include/python3.9', '-DPYTHONDIR="/usr/local/lib/python3.9/site-packages"', '-DPYTHONARCHDIR="/usr/local/lib/python3.9/site-packages"', '-DNDEBUG', '../test.cpp', '-c', '-o/build/wxPython-4.1.1/build/waf/3.9/gtk3/.conf_check_eaab107a0ad0c09fe99b0e743a890f00/testbuild/test.cpp.1.o']
  00:05:56 runner ['g++', '-shared', 'test.cpp.1.o', '-o/build/wxPython-4.1.1/build/waf/3.9/gtk3/.conf_check_eaab107a0ad0c09fe99b0e743a890f00/testbuild/testprog.cpython-39-x86_64-linux-gnu.so', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/nix/store/nki9ywqzbvz68vr75kn2r7g1q84f5agy-python3-3.9.6/lib', '-lpthread', '-ldl', '-lcrypt', '-lncurses', '-lutil', '-lm', '-lm', '-lpthread', '-ldl', '-lcrypt', '-lncurses', '-lutil', '-lm', '-lm']
  yes 
  Command '/build/wxPython-4.1.1/build/wxbld/gtk3/wx-config --cflags' failed with exit code 127.
  /nix/store/dpjnjrqbgbm8a5wvi1hya01vd8wyvsq4-bash-4.4-p23/bin/sh: /build/wxPython-4.1.1/build/wxbld/gtk3/wx-config: No such file or directory
  Command '"/nix/store/nki9ywqzbvz68vr75kn2r7g1q84f5agy-python3-3.9.6/bin/python3.9" /build/wxPython-4.1.1/bin/waf-2.0.19 --verbose --wx_config=/build/wxPython-4.1.1/build/wxbld/gtk3/wx-config --verbose --gtk3 --python="/nix/store/nki9ywqzbvz68vr75kn2r7g1q84f5agy-python3-3.9.6/bin/python3.9" --out=build/waf/3.9/gtk3 configure build ' failed with exit code 127.
  Finished command: build_py (0m2.171s)
[0 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/h1q5d9jj2x1yghbc0d4i04yva25gyqc8-python3.9-wxPython-4.1.1.drv' failed

tfmoraes · 2021-08-25T01:07:57Z

I did some modifications and it worked, see: https://gist.github.com/tfmoraes/fc2f1b0e4d571f5c92754720a8303d6b

davidak · 2021-08-25T01:32:42Z

It builds, but does not work.

building '/nix/store/q781h2mz5pwgvcmg4wwhkly43llc85zp-python3-3.9.6-env.drv'...
created 283 symlinks in user environment
Python 3.9.6 (default, Jun 28 2021, 08:57:49) 
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/187bwr9g29405apfm7l6hr3jcsdbcfkm-python3-3.9.6-env/lib/python3.9/site-packages/wx/__init__.py", line 17, in <module>
    from wx.core import *
  File "/nix/store/187bwr9g29405apfm7l6hr3jcsdbcfkm-python3-3.9.6-env/lib/python3.9/site-packages/wx/core.py", line 12, in <module>
    from ._core import *
ImportError: libwx_gtk3u_core-3.1.so.5: cannot open shared object file: No such file or directory

tfmoraes · 2021-08-25T02:29:14Z

Adding autoPatchelfHook to nativeBuildInputs fix this problem. I don't know if this is the better thing to do. Other option, I think, is use patchelf to set rpath to $ORIGIN to the problematic elf files.

davidak · 2021-08-25T04:45:34Z

I think autoPatchelfHook is an elegant and stable solution.

It works now 🚀

pkgs/development/python-modules/wxPython/4.1.nix

SuperSandro2000 · 2021-08-25T11:39:03Z

Adding autoPatchelfHook to nativeBuildInputs fix this problem. I don't know if this is the better thing to do. Other option, I think, is use patchelf to set rpath to $ORIGIN to the problematic elf files.

Manually patch it or even better build from source which I think is the right choice here.

Co-authored-by: Thiago Franco de Moraes <totonixsame@gmail.com>

davidak · 2021-08-25T16:53:50Z

build from source which I think is the right choice here.

i don't understand where the file comes from. do they distribute binary files? so i'm just happy when the package is updated. others can improve it further

tfmoraes · 2021-08-25T19:15:39Z

Those files are compiled from source on compiling process of wxPython. In the compiling process those file should have set the rpath, but that is not happening. Something like in this bug report wxWidgets/Phoenix#423

tfmoraes · 2021-08-25T19:31:59Z

I checked here. wxPython compiled on Ubuntu appends $ORIGIN to RUNPATH:

$ LC_ALL=C readelf -d wx/_core.cpython-38-x86_64-linux-gnu.so | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN]

That is not the case of the version compile on NixOS (without the autoPatchelfHook):

$ LC_ALL=C readelf -d  result/lib/python3.9/site-packages/wx/_core.cpython-39-x86_64-linux-gnu.so | grep RUNPATH                                                        
 0x000000000000001d (RUNPATH)            Library runpath: [/nix/store/9bh3986bpragfjmr32gay8p95k91q4gy-glibc-2.33-47/lib:/nix/store/0kiykyrnrpfhmjwxwx89kxr20hmf5304-gcc-10.3.0-lib/lib]

SuperSandro2000 · 2021-08-25T22:23:24Z

This means we compile the binary but the RPATH is not set correct, right?

If so please comment that if not already done.

tfmoraes · 2021-08-26T01:45:15Z

This means we compile the binary but the RPATH is not set correct, right?

Yes.

See NixOS/nixpkgs#135607

tfmoraes · 2022-02-18T23:04:18Z

I think this PR can be merged. It's working in my tests.

SuperSandro2000 · 2022-02-20T01:02:45Z

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).

Result of nixpkgs-review pr 135607 run on x86_64-linux 1

1 package failed to build and already failed to build on hydra master:

playonlinux: plain log | pretty log

2 packages failed to build and are new build failures:

kicad-unstable-small: log was empty
python38Packages.wxPython_4_1: log was empty

1 package built:

python39Packages.wxPython_4_1

sigprof · 2022-02-21T14:28:21Z

Looks like this change broke the build of the stable KiCad version too:
https://hydra.nixos.org/build/167894687

-- Found Phoenix 4.1.1/gtk3 (wxWidgets 3.1.5)
CMake Error at /nix/store/sgsvy949kd2sr2vsxcffz5jkv855rmib-cmake-3.22.2/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find wxWidgets: Found unsuitable version "3.1.4", but required is
  at least "3.1.5" (found
  -L/nix/store/pcc4pldpa2xwqh65gazrlvv2z5y9xmbs-wxwidgets-3.1.4/lib;-pthread;;;-L/nix/store/5lzlfgrc37wqj400b4qgpaa6kc9d2wr1-libglvnd-1.4.0/lib;-L/nix/store/04yj0nllw5bqzzlay5m6zzbmyw7csgnk-glu-9.0.2/lib;-lwx_gtk3u_gl-3.1;-lwx_gtk3u_aui-3.1;-lwx_gtk3u_html-3.1;-lwx_gtk3u_core-3.1;-lwx_baseu_net-3.1;-lwx_baseu-3.1;-lwx_gtk3u_propgrid-3.1;-lwx_baseu_xml-3.1;-lwx_gtk3u_stc-3.1;-lwx_gtk3u_richtext-3.1)
Call Stack (most recent call first):
  /nix/store/sgsvy949kd2sr2vsxcffz5jkv855rmib-cmake-3.22.2/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:592 (_FPHSA_FAILURE_MESSAGE)
  CMakeModules/FindwxWidgets.cmake:1019 (find_package_handle_standard_args)
  CMakeLists.txt:876 (find_package)

KiCad uses wxWidgets both directly and through wxPython, therefore changing wxPython to use the bundled newer version of wxWidgets breaks the kicad-base package.

@evils I suppose that even reviving #98634 won't work to fix this from the KiCad side, because the updated wxPython package no longer exposes the underlying wxGTK.

davidak · 2022-02-21T14:56:11Z

@sigprof Maybe there is an upstream issue for this situation or we can look how other distros handle it?

evils · 2022-02-21T16:33:02Z

kicad and kicad-unstable build with wxPython 4.1.1 and wxGTK31 3.1.5
so maybe waiting for https://nixpk.gs/pr-tracker.html?pr=157848 will suffice?

sigprof · 2022-02-22T06:03:05Z

pkgs/development/python-modules/wxPython/4.1.nix

    substituteInPlace wx/lib/wxcairo/wx_pycairo.py \
      --replace '_dlls = dict()' '_dlls = {k: ctypes.CDLL(v) for k, v in [
        ("gdk",        "${wxGTK.gtk}/lib/libgtk-x11-3.0.so"),
        ("pangocairo", "${pango.out}/lib/libpangocairo-1.0.so"),
+        ("cairoLib = None", "cairoLib = ctypes.CDLL('${cairo}/lib/libcairo.so')"),


This part does not seem to be correct — the text ends up being included in the Python code, while the original suggestion was to add a pair of arguments to substituteInPlace, so that one call will perform two replacements in separate parts of the same file.

sigprof · 2022-02-22T06:44:21Z

Maybe there is an upstream issue for this situation or we can look how other distros handle it?

wxWidgets/Phoenix#1961 suggests making a wxPython release that can build with system wxWidgets 3.1.5, but apparently this requires applying some patches to wxWidgets and won't work with an unmodified wxWidgets 3.1.5 release. However, a more traditional distro would probably need to do exactly that, and even NixOS may need to obey similar restrictions if there is some chance that different versions of the same shared library may end up in the dependency tree of a single executable.

kicad and kicad-unstable build with wxPython 4.1.1 and wxGTK31 3.1.5

The existence of wxGTK31 3.1.5 probably satisfies CMake, but the resulting binaries will still have two different wxGTK versions in the dependency tree — one as a direct shared library dependency, another as a dependency of wxPython shared modules. But this may be OK if the Python code executed by the embedded Python interpreter never actually imports any wxPython modules. I'm not familiar with KiCad internals — does it always spawn a separate process to run any Python code that might use wxPython, or may it end up running some Python code of that kind in the process that also uses wxGTK directly for its GUI?

(Actually even KiCad packages built before this PR depend on two different builds of wxWidgets — the directly used wxGTK31 with the default options and the indirect dependency through wxPython which is built with withWebKit = true; this might be safe if enabling WebKit just adds an extra shared library without changing the ABI of any other wxGTK libraries, but still is not ideal in terms of the closure size.)

evils · 2022-02-22T11:16:25Z

oh indeed, kicad's python terminal (in the PCB editor) fails to open due to something related to wxpython 4.1.1

i think the only right solution is for wxpython phoenix to switch to using upstream releases of wxwidgets
and they seem quite reluctant to do so (for understandable reasons...)

sigprof · 2022-02-22T14:13:36Z

Arch Linux reverted to old wxPython/wxWidgets combination because of KiCad build issues — currently they have wxPython 4.0.7.2 and wxGTK 3.0.5.1. In theory the wxGTK 3.0 branch is “stable”, but it's also rather old, and the corresponding wxPython version is really old too.

The Flatpak for KiCad apparently uses unmodified wxWidgets-3.1.5 and wxPython-4.1.1 patched with fix-wxpython-4.1.1-on-wxwidgets-3.1.5.patch; that patch apparently comes from wxWidgets/Phoenix#1909 (comment). So waiting for #157848 and then updating wxPython to build with that version of wxGTK31 using that patch may be a suitable solution. I don't know how important are the other patches mentioned in wxWidgets/Phoenix#1961 though.

evils · 2022-02-22T17:00:24Z

an approach that's more in line with wxPython phoenix's apparently intended usage is to have the wxPython package expose the wxwidgets it was built with (the one they bundle in their source)
and then use that wxwidgets everywhere wxPython and wxGTK are used together (just kicad and grass?)
but that would bring the amount of packages that provide a wxwidgets output to 7

edit: it was mentioned above that

the updated wxPython package no longer exposes the underlying wxGTK.

which would mean this is indeed not an option...

github-actions bot added the 6.topic: python label Aug 24, 2021

ofborg bot requested a review from tfmoraes August 24, 2021 22:54

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Aug 24, 2021

davidak force-pushed the wxPython branch from fb22226 to 51744f0 Compare August 25, 2021 02:06

davidak force-pushed the wxPython branch from 51744f0 to 833982b Compare August 25, 2021 04:43

davidak marked this pull request as ready for review August 25, 2021 04:45

davidak requested review from FRidh and jonringer as code owners August 25, 2021 04:45

SuperSandro2000 reviewed Aug 25, 2021

View reviewed changes

pkgs/development/python-modules/wxPython/4.1.nix Show resolved Hide resolved

wxPython: 4.1.0 -> 4.1.1

e559b4e

Co-authored-by: Thiago Franco de Moraes <totonixsame@gmail.com>

davidak force-pushed the wxPython branch from 833982b to e559b4e Compare August 25, 2021 16:53

tfmoraes approved these changes Aug 30, 2021

View reviewed changes

plabadens added a commit to plabadens/nur-packages that referenced this pull request Dec 12, 2021

wxPython: 4.1.0 -> 4.1.1

afc7893

See NixOS/nixpkgs#135607

SuperSandro2000 merged commit 4f2404d into NixOS:master Feb 20, 2022

davidak deleted the wxPython branch February 20, 2022 09:49

sigprof reviewed Feb 22, 2022

View reviewed changes

evils mentioned this pull request Feb 22, 2022

issue collection/overview of wxWidgets, wxGTK, wxPython (Phoenix), wxmac #90654

Open

This was referenced Feb 24, 2022

wxwidgets: 3.1.4 -> 3.1.5 #161593

Closed

kicad: 6.0.1 -> 6.0.2 #159132

Merged

davidak mentioned this pull request Mar 3, 2022

timeline: 2.4.0 -> 2.6.0, fix build #162503

Merged

13 tasks

evils mentioned this pull request Mar 20, 2022

kicad: 6.0.2 -> 6.0.4 #164981

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wxPython: 4.1.0 -> 4.1.1 #135607

wxPython: 4.1.0 -> 4.1.1 #135607

davidak commented Aug 24, 2021 •

edited

Loading

tfmoraes commented Aug 24, 2021

davidak commented Aug 25, 2021 •

edited

Loading

tfmoraes commented Aug 25, 2021

davidak commented Aug 25, 2021

tfmoraes commented Aug 25, 2021

davidak commented Aug 25, 2021

SuperSandro2000 commented Aug 25, 2021 •

edited

Loading

davidak commented Aug 25, 2021

tfmoraes commented Aug 25, 2021

tfmoraes commented Aug 25, 2021

SuperSandro2000 commented Aug 25, 2021 •

edited

Loading

tfmoraes commented Aug 26, 2021

tfmoraes commented Feb 18, 2022

SuperSandro2000 commented Feb 20, 2022

sigprof commented Feb 21, 2022

davidak commented Feb 21, 2022

evils commented Feb 21, 2022 •

edited

Loading

sigprof Feb 22, 2022

sigprof commented Feb 22, 2022

evils commented Feb 22, 2022

sigprof commented Feb 22, 2022

evils commented Feb 22, 2022 •

edited

Loading

wxPython: 4.1.0 -> 4.1.1 #135607

wxPython: 4.1.0 -> 4.1.1 #135607

Conversation

davidak commented Aug 24, 2021 • edited Loading

Motivation for this change

Things done

tfmoraes commented Aug 24, 2021

davidak commented Aug 25, 2021 • edited Loading

tfmoraes commented Aug 25, 2021

davidak commented Aug 25, 2021

tfmoraes commented Aug 25, 2021

davidak commented Aug 25, 2021

SuperSandro2000 commented Aug 25, 2021 • edited Loading

davidak commented Aug 25, 2021

tfmoraes commented Aug 25, 2021

tfmoraes commented Aug 25, 2021

SuperSandro2000 commented Aug 25, 2021 • edited Loading

tfmoraes commented Aug 26, 2021

tfmoraes commented Feb 18, 2022

SuperSandro2000 commented Feb 20, 2022

sigprof commented Feb 21, 2022

davidak commented Feb 21, 2022

evils commented Feb 21, 2022 • edited Loading

sigprof Feb 22, 2022

Choose a reason for hiding this comment

sigprof commented Feb 22, 2022

evils commented Feb 22, 2022

sigprof commented Feb 22, 2022

evils commented Feb 22, 2022 • edited Loading

davidak commented Aug 24, 2021 •

edited

Loading

davidak commented Aug 25, 2021 •

edited

Loading

SuperSandro2000 commented Aug 25, 2021 •

edited

Loading

SuperSandro2000 commented Aug 25, 2021 •

edited

Loading

evils commented Feb 21, 2022 •

edited

Loading

evils commented Feb 22, 2022 •

edited

Loading