Segfault on crossing python<->c boundary.

[HinTak]

I am having problem with a segfault which I don't know where to look for. It seems to be segfault'tiing in code which is not currently explicitly running, so it sounds like a destructor / reference count issue. Here is the GDB trace:

#0  0x0000000000427621 in ?? ()
#1  0x00007fffe7b09504 in ?? ()
   from /home/Hin-Tak/.local/lib/python3.11/site-packages/skia_python-117.0b2-py3.11-linux-x86_64.egg/skia.cpython-311-x86_64-linux-gnu.so
#2  0x00007fffe7acb11b in ?? ()
   from /home/Hin-Tak/.local/lib/python3.11/site-packages/skia_python-117.0b2-py3.11-linux-x86_64.egg/skia.cpython-311-x86_64-linux-gnu.so
#3  0x00007fffe7acae38 in ?? ()
   from /home/Hin-Tak/.local/lib/python3.11/site-packages/skia_python-117.0b2-py3.11-linux-x86_64.egg/skia.cpython-311-x86_64-linux-gnu.so
#4  0x00007ffff7bc7197 in _PyFrame_Clear () from /lib64/libpython3.11.so.1.0
#5  0x00007ffff7bb5379 in _PyEval_Vector () from /lib64/libpython3.11.so.1.0
#6  0x00007ffff7bc68a7 in PyObject_Vectorcall () from /lib64/libpython3.11.so.1.0
#7  0x00007ffff7f72047 in _CallPythonObject () from /usr/lib64/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so
#8  0x00007ffff7f72197 in closure_fcn () from /usr/lib64/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so
#9  0x00007ffff7f93711 in ffi_closure_unix64_inner () from /lib64/libffi.so.8
#10 0x00007ffff7f93e08 in ffi_closure_unix64 () from /lib64/libffi.so.8
#11 0x00007fffe9b324c4 in ft_svg_render.lto_priv () from /lib64/libfreetype.so.6
#12 0x00007fffe9ac677b in FT_Render_Glyph_Internal () from /lib64/libfreetype.so.6
#13 0x00007ffff7f93be6 in ffi_call_unix64 () from /lib64/libffi.so.8
#14 0x00007ffff7f904bf in ffi_call_int.lto_priv () from /lib64/libffi.so.8
#15 0x00007ffff7f9318e in ffi_call () from /lib64/libffi.so.8
#16 0x00007ffff7f6ae63 in _ctypes_callproc.cold () from /usr/lib64/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so
#17 0x00007ffff7f6a7bf in PyCFuncPtr_call.cold () from /usr/lib64/python3.11/lib-dynload/_ctypes.cpython-311-x86_64-linux-gnu.so
#18 0x00007ffff7bb0953 in _PyObject_MakeTpCall () from /lib64/libpython3.11.so.1.0
#19 0x00007ffff7bb92a8 in _PyEval_EvalFrameDefault () from /lib64/libpython3.11.so.1.0
#20 0x00007ffff7bb536a in _PyEval_Vector () from /lib64/libpython3.11.so.1.0
#21 0x00007ffff7c397ac in PyEval_EvalCode () from /lib64/libpython3.11.so.1.0
#22 0x00007ffff7c57053 in run_eval_code_obj () from /lib64/libpython3.11.so.1.0
#23 0x00007ffff7c536ea in run_mod () from /lib64/libpython3.11.so.1.0
#24 0x00007ffff7c693f2 in pyrun_file () from /lib64/libpython3.11.so.1.0
#25 0x00007ffff7c68ba8 in _PyRun_SimpleFileObject () from /lib64/libpython3.11.so.1.0
#26 0x00007ffff7c68808 in _PyRun_AnyFileObject () from /lib64/libpython3.11.so.1.0
#27 0x00007ffff7c6274c in Py_RunMain () from /lib64/libpython3.11.so.1.0
#28 0x00007ffff7c2977b in Py_BytesMain () from /lib64/libpython3.11.so.1.0
--Type <RET> for more, q to quit, c to continue without paging--
#29 0x00007ffff7849b4a in __libc_start_call_main () from /lib64/libc.so.6
#30 0x00007ffff7849c0b in __libc_start_main_impl () from /lib64/libc.so.6
#31 0x0000555555555095 in _start ()
(gdb) 

What I am doing in this script, is having registered some callback function via ctypes, the callback function runs and seems to be returning to the C-side successfully. However, it then somehow dies and segfaults in the pybind11-code.

The callback is a ctypes callback, but it creates some pybind11 objects, and hope to hang onto them until the next callback. There are two ctypes callbacks, the first interacts with pybind11 somewhat, and hopes to keep some of the objects (via assigning to global, etc) until the 2nd ctypes callback, if that makes sense?

The first callback is fully successful and returns, but then the script segfaults with
this stack trace.

[HinTak]

I couldn't find any info on _PyFrame_Clear - not even on cpython's source code.

[HinTak]

To answer my own question - I was missing a return_value_policy::reference_internal in one of the newly added routines. Still don't understand it, but at least it is a direction I can go to see whether that's really the answer, and how does it work.