MacOS codesign fails in tcltk subcomponent: 'bundle format unrecognized, invalid, or unsuitable'

[Willy-JL]

The build workflow for my app was going just fine on Windows, Linux and macOS with GitHub actions, then I made some changes to some code and wanted a new binary build. Nothing that I changed I feel like was significant to requiring more packages or influencing the build process, but still it started failing for macOS. The last successful build I have is from 23/10/2022, then next build I tried is from 25/10/2022 and that started failing. Haven't had a single successful build since then.

So at first it failed because libtcl8.6.dylib was missing. Turns out the macos-latest runner got updated to macos-12 and that had a bug, with tcl missing. That was since fixed. Then just for the hell of it I tried python 3.11, but that couldn't build one of my requirements because of outdated cython. I updated that and now the requirement built fine on 3.11. So I tried another build, and now we get to the error I'm stuck on:

error: [Errno 2] No such file or directory: '/usr/local/opt/libiodbc/lib/libiodbc.2.dylib'

Tried installing cx_Freeze in many different ways, from pypi, source (tag 6.13.0), latest dev packages, and it would either fail to install cx_Logging, or it would get the above error while building my app. So then I tried to install IODBC to make sure libiodbc.2.dylib is present, but the brew install libiodbc command fails because of conflicts with unixiodbc because both provide odbcinst.h. Finally decided to give up on python 3.11 and revert back to 3.10.8. Tried the same troubleshooting attempts, same results. So I finally thought: it has to be another issue with macos-12, but that wasn't it. I force downgraded to the macos-11 runner, and still libiodbc.2.dylib is missing. And its not an update to the runner that caused it because it used the same exact macos-11 runner image version (20221018.1) that worked on 23/10/2022.

My best guess at this point is that one of my requirements, or their dependencies, received an update and that makes it fail now. Do you have any tips on how I could figure this out?

[marcelotduarte]

Do you need the libiodbc in your app? If it is dependency for some package, but you don't use, maybe you can use bin_excludes.
If you need it, check what is the package that use it, and downgrade the package.

cx_Logging it not used by cx_Freeze installed in macOS.

[Willy-JL]

I don't even know what libiodbc is to be perfectly honest. I checked previous working builds and there is nothing related to it anywhere to be found. I tried adding iodbc to bin_excludes, but that didn't work. tried with libiodbc in bin_excludes, also didn't work. Lowered the silence level to have some more info and I get this:

...
copying /Users/runner/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/PyQt6/Qt6/plugins/sqldrivers/libqsqlodbc.dylib -> /Users/runner/work/App/App/build/exe.macosx-10.15-x86_64-3.10/lib/PyQt6/Qt6/plugins/sqldrivers/libqsqlodbc.dylib
copying /usr/local/opt/libiodbc/lib/libiodbc.2.dylib -> /Users/runner/work/App/App/build/exe.macosx-10.15-x86_64-3.10/libiodbc.2.dylib
error: [Errno 2] No such file or directory: '/usr/local/opt/libiodbc/lib/libiodbc.2.dylib'
Error: Process completed with exit code 1.

How exactly do I got about excluding it if bin_excludes doesn't do it? Or am I using it wrong? Otherwise, how can I find out what package it belongs to?

[marcelotduarte]

Use the complete name, bin_excludes=["libqsqlodbc.dylib"] like we do in samples for another issue:
https://github.com/marcelotduarte/cx_Freeze/blob/main/samples/pyside6/setup.py#L47

[Willy-JL]

To be fair I'm quite certain this is a regression problem. I just now tried levelling the playing field: ran 2 tests, both on macos-12, both WITHOUT any bin_excludes, but using different cx_Freeze versions. v6.11.1 builds perfectly as intended, 6.13.1 fails with the missing libiodbc error. If i add the bin_excludes then another missing library error, and adding that too then it builds fine on 6.13.1. However if I then try to codesign that, I get a bundle format unrecognized, invalid, or unsuitable error with App.app/Contents/MacOS/lib/tcltk/tk8.6. Either way my point is that something probably went wrong between v6.11.1 and 6.13.1 that now causes some pretty serious library inclusion problems.

My reconstruction of the facts I went through is that when macos-latest switched to macos-12, the new image had a bug that failed the build, and at first i tried updating cx_Freeze to solve it. Then, when the macos-12 bug was fixed, I still kept using the newer version of cx_Freeze and kept getting this error.

EDIT: actually can confirm that this issue is present as far back as 6.12.0, so something went wrong with that update in particular IMO, from v6.11.1 to 6.12.0.

[Willy-JL]

And today I tried again with 6.13.1 and it built fine. This is beyond me, I'm assuming all the issues I was having above were with the runner image and not with cx_Freeze, but the original issue is fixed.

However a new issue arose: after the successful build, codesign doesnt work because something is wrong with tcltk:

$ codesign -s "***" --deep ./App.app
./F95Checker.app: bundle format unrecognized, invalid, or unsuitable
In subcomponent: /Users/runner/work/F95Checker/F95Checker/dist/App.app/Contents/MacOS/lib/tcltk/tk8.6

v6.11.1 builds and codesigns, I ran both on the same runner images so the libraries present on the system are the same...

I "solved" it by excluding tkinter, but that's not really a solution...

[marcelotduarte]

Try to install using:
pip install --no-binary=cx_Freeze cx_Freeze

[Willy-JL]

I was already installing from source at tag 6.13.1. Anyway I tried that. Same error. Also tried installing from main branch, same error.

[Willy-JL]

Tried downgrading from 3.11 to python 3.10.8, still same error.

I personally don't need tkinter, but I found this problem and thought I'd report it. Sorry I can't be of much help

[vasole]

I arrived to this topic because I was getting the same original error: [Errno 2] No such file or directory: '/usr/local/opt/libiodbc/lib/libiodbc.2.dylib'. I can confirm that the error is not present with cx_Freeze 6.11.1 as written above.

[marcelotduarte]

Use from source dist:
pip install --no-binary=cx_Freeze cx_Freeze

[vasole]

Also tried (I read the topic prior to post).

[marcelotduarte]

Also, use excludes for some modules, like tkinter, and issue #1511 can be relevant.

[vasole]

tkinter is excluded. I'll take (another) look at #1511 although I am not on an M1 machine.

[vasole]

For your information: I have located the missing library as /usr/lib/libiodbc.2.dylib

[marcelotduarte]

If you use bin_path_includes or bin_includes or both, do you get success?

[vasole]

Is there a way to add a path without losing the others? If I set bin_path_includes to ['/usr/lib'] then there are other libraries missing.

To use bin_includes to specify the full path to that particular library does not work either. cx_Freeze still tries to fetch it from /usr/local/opt/libiodbc/lib

[vasole]

Somehow I have the same findings as @Willy-JL

EDIT: actually can confirm that this issue is present as far back as 6.12.0, so something went wrong with that update in particular IMO, from v6.11.1 to 6.12.0.

[vasole]

Just in case it helps, I suspect a different handling of the PySide6/PyQt5/PyQt6 QtSql module between 6.11.1 and the subsequent versions.

If I set build_options["bin_excludes"] = ["libiodbc", "libiodbc.2.dylib", "libpq.5.dylib"] the application is properly frozen with latest cx_Freeze.

[marcelotduarte]

If you do not use QtSql, you can add excludes PySide2.QtSql (are you using pyside2?)
This should block the finder to search for the shared libraries that it depends on.
I do see changes in the handling, but there are a lot of changes in the code in general.

[vasole]

If you do not use QtSql, you can add excludes PySide2.QtSql (are you using pyside2?)

Tried without success.

[marcelotduarte]

Release 6.15.3 is out!

Can you test it and give me feedback?

[vasole]

Release 6.15.3 is out!

Can you test it and give me feedback?

I do not know if the question was addressed to me or to other poster.

Nothing changes for me. Adding PyQt5.QtSql and PySide6.QtSql to the excludes list does not work. I need to explicitly exclude the offending libraries as mentioned above.

[TechnicalPirate]

Thanks for linking me this Marcel

I can confirm the same as the above, with cx_Freeze-6.15.4 i need to set

"bin_excludes": ["libiodbc.2.dylib", "libpq.5.dylib"]

in the build_exe options as a workaround -> after doing this I'm encountering code-sign issues i suspect the same as others have reported but have an issue with CI i need to fix before i can verify. Actually I'm getting

stderr: build/simple_PyQt5-0.1.app: is already signed

... when i try and sign the output 🤔

When i run codesign verification i get

stderr: build/simple_PyQt5-0.1.app: invalid signature (code or signature have been modified)

Last edit for today: Could this PR be related?

[TechnicalPirate]

Todays updates :)

I set codesign to --force to workaround the already signed issue. Upon doing this i hit the invalid bundle structure error(s)

[2023-07-31T09:27:45.117Z] Codesigning the bundle.
[2023-07-31T09:30:54.830Z]  stderr: build/simple_PyQt5-0.1.app: replacing existing signature
[2023-07-31T09:30:54.830Z] build/simple_PyQt5-0.1.app: bundle format unrecognized, invalid, or unsuitable
[2023-07-31T09:30:54.830Z] In subcomponent: /Volumes/jenkins/workspace/MyAppBuild-api_PR-10/MyAppBuild/samples/PySide2/build/simple_PyQt5-0.1.app/Contents/MacOS/lib/PySide2/Qt/qml/QtQml/Models.2

at this point i started re-enabling functionality from the gist provided in #594 - This bypassed the bundle structure issues 🎉reported by codesign...

...but - By the end of day ( time of writing ) - I still have yet to get a build passing notarization - Most the Qt library resources are still throwing Binary signature warnings for example

            "path": "sampleqt-0.1.057d4b5.pkg/sampleqt-0.1.057d4b5.pkg Contents/Payload/Applications/sampleqt.app/Contents/Resources/lib/QtWebSockets",
            "message": "The binary is not signed.",
            "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087721",
            "architecture": "x86_64"

and i still get a couple of errors for the Python binary itself

        {
            "severity": "error",
            "code": null,
            "path": "sampleqt-0.1.057d4b5.pkg/sampleqt-0.1.057d4b5.pkg Contents/Payload/Applications/sampleqt.app/Contents/Resources/lib/Python",
            "message": "The binary is not signed with a valid Developer ID certificate.",
            "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087721",
            "architecture": "arm64"
        },

... i suspect this is due to some incompatible changes between cx_Freeze and our wrapper and its not nuking the qt libraries we don't need.

I will continue pulling on this thread tomorrow

TLDR:

• bin_excludes needs to be set for specific .dylibs
• the _restructure_app_bundle code from the linked jist seems legit for resolving the invalid bundle structure errors
• the output currently still isn't passing notarization - (Investigations continue...)

Again, my hope is once i get a sample building we can dissect the gist / contribute back the workarounds :)

[TechnicalPirate]

Todays update:

• Saw the cx_freeze-6.15.5 release and updated to that ( thanks! )
  • Disabled custom code again to test native output of .5
  • Continued to hit the issue with QtQml/Models.2 throwing bundle structure errors
  • Discovered this is related to the .2 directory -> discussions here
    • PyInstaller recipe
    • PyInstaller issue
• Continued selectively testing parts of our existing restructuring code. - Goal remains the same :)

[marcelotduarte]

@TechnicalPirate
When I tested a recent patch to get pyqt5 working on conda Linux I also tried it on other systems/environments.
I concluded that a frozen pyqt5 app works fine on Windows and Linux using pip, and it works fine on macOS using conda. It might be worth a try.

[TechnicalPirate]

Hi @marcelotduarte !

Mini update as i kept tinkering:

• Sorry I'm afraid i don't quite understand your meaning in the above comment 😅
• Good news, setting "zip_include_packages": ["PySide2", "shiboken2"] in the build_exe options as well as ignoring the problematic .dylibs seems to sidestep this issue!
• That said, still getting some issues with the output - But i'll take the discussion to Mac OS bundle are rejected by Apple's notarization process  #594 as that issue better reflects apples expectations for product structure
• I was looking for documentation on the zip_include_packages option but couldn't find anything beyond the flag settings, can you explain a bit the intended usage / use cases / what the trade offs of using it are?

[marcelotduarte]

• Good news, setting "zip_include_packages": ["PySide2", "shiboken2"] in the build_exe options seems to sidestep this issue!

Some time ago, I discovered that some users used the option zip_include_packages to reduce the files copied to frozen binary. So I overloaded it to get a optimized mode. The original objective is to reduce the files, copying only the used modules (.py) to library.zip, and I overloaded to copy only the used files (modules .py, extensions .so, etc).
The sample suggests this use.

Sorry I'm afraid i don't quite understand your meaning in the above comment 😅

I guess that you uses pyqt5, but for pyside2 is the same. I mean that when I make a patch to fix bug or improvement, I test, using the sample, the various OS (Windows, Linux, macOS) via CI, to check if that are regressions. With recent versions of macOS, using pip to install pyqt5 (and pyside2), this is broken. But using the conda on macOS the sample works.

[marcelotduarte]

Hi @vasole @Willy-JL

Nothing changes for me. Adding PyQt5.QtSql and PySide6.QtSql to the excludes list does not work. I need to explicitly exclude the offending libraries as mentioned above.

With the sample pyside6 and all others using Qt, I cannot reproduce this error. Can you adapt a sample to reproduce?

I tested with latest (6.15.10) and with development version:
pip install --force --no-cache --pre --extra-index-url https://marcelotduarte.github.io/packages/ cx_Freeze