PR: Remove macOS app flow fork in closing Spyder #17732

mrclary · 2022-04-22T03:27:46Z

Description of Changes

Closing the Spyder application in mainwindow.py has a flow control fork for the macOS standalone application. This is removed.

Issue(s) Resolved

Fixes #17729
Partial fix for #17731

Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: @mrclary

mrclary · 2022-04-22T03:38:53Z

@ccordoba12, after removing the flow control for the macOS app in the artifact from #16409 I was unable to reproduce any segfaults. Perhaps the segfaults were encountered earlier in your development for that PR and incidentally resolved in the same PR after inserting the flow control?

Nevertheless, when testing on this PR with local builds, closing the app does not produce any errors or faults and all Python processes are exited. Restarting Spyder from the menubar get much further now, but still fails. The restarting splash screen appears, as well as the new starting splash screen, but then fails with the following error:

Traceback (most recent call last):
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/start.py", line 254, in <module>
    main()
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/start.py", line 250, in main
    mainwindow.main(options, args)
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/mainwindow.py", line 2057, in main
    mainwindow = create_window(MainWindow, app, splash, options, args)
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/utils.py", line 289, in create_window
    main.setup()
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/mainwindow.py", line 923, in setup
    mainmenu = self.mainmenu
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/mainwindow.py", line 1069, in __getattr__
    return super().__getattr__(attr)
AttributeError: 'MainWindow' object has no attribute 'mainmenu'

mrclary · 2022-04-22T17:28:21Z

The restart sequence launches a subprocess that runs restart.py, which in turn launches a subprocess that runs start.py. Recreating the command and environment variables of the subprocess launched by restart.py I can reproduce the error from the command line.

Relevant environment variables:

SPYDER_DEBUG_FILE=/Users/rclary/.spyder-py3-dev/spyder-debug.log
QT_MAC_WANTS_LAYER=1
PYTHONUNBUFFERED=1
SPYDER_DEBUG=2
RESOURCEPATH=/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources
_PY2APP_LAUNCHED_=1
QT_SCREEN_SCALE_FACTORS=
EVENT_NOKQUEUE=1
PYTHONHOME=/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources
PYTHONDONTWRITEBYTECODE=1
QT_API=pyqt5
ARGVZERO=dist/Spyder.app/Contents/MacOS/Spyder
EXECUTABLEPATH=/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/MacOS/Spyder
QT_SCALE_FACTOR=

Command:

Spyder.app/Contents/MacOS/python Spyder.app/Contents/Resources/lib/python3.9/spyder/app/start.py --new-instance

Output:

The available OpenGL surface format was either not version 3.2 or higher or not a Core Profile.
Chromium on macOS will fall back to software rendering in this case.
Hardware acceleration and features such as WebGL will not be available.
fromIccProfile: failed minimal tag size sanity
2022-04-22 10:13:10,805 [INFO] [spyder.app.mainwindow] -> Start of MainWindow constructor
2022-04-22 10:13:10,806 [INFO] [spyder.app.mainwindow] -> End of MainWindow constructor
2022-04-22 10:13:10,806 [INFO] [spyder.app.mainwindow] -> *** Start of MainWindow setup ***
2022-04-22 10:13:10,806 [INFO] [spyder.app.mainwindow] -> Updating PYTHONPATH
2022-04-22 10:13:10,806 [INFO] [spyder.app.mainwindow] -> Applying theme configuration...
2022-04-22 10:13:10,821 [INFO] [spyder.app.mainwindow] -> Loading switcher...
2022-04-22 10:13:10,826 [INFO] [spyder.app.mainwindow] -> Loading old third-party plugins...
2022-04-22 10:13:10,955 [INFO] [spyder.app.mainwindow] -> Creating Menus...
Traceback (most recent call last):
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/start.py", line 254, in <module>
    main()
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/start.py", line 250, in main
    mainwindow.main(options, args)
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/mainwindow.py", line 2028, in main
    mainwindow = create_window(MainWindow, app, splash, options, args)
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/utils.py", line 289, in create_window
    main.setup()
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/mainwindow.py", line 926, in setup
    mainmenu = self.mainmenu
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/mainwindow.py", line 1072, in __getattr__
    return super().__getattr__(attr)
AttributeError: 'MainWindow' object has no attribute 'mainmenu'

I'll investigate further what seems to be causing a problem for the macOS app. Maybe it's an environment variable, or maybe it's something in start.py. But the error indicates that the mainmenu plugin is not being created and/or not added to the MainWindow object.

I also know that the Spyder macOS app can be launched from the command line using Spyder.app/Contents/MacOS/Spyder [options], so this may be an alternative command to start.py for the standalone application.

mrclary · 2022-04-22T19:39:27Z

Okay, the issue seems to be that pkg_resources is not loading the entry points when launched from restart.py. However, this is not the case if I attempt to do so from the artifact of #17409. So I think that removing the flow control fork for the macOS app does resolve this and #17729. At some later, yet to be determined commit, pkg_resources does not load the entry points upon restart. This may have to do with the updates to py2app...

mrclary · 2022-04-22T20:10:11Z

Okay, I've narrowed it down to #17612, the update to py2app=0.28. Now I will check whether this has been fixed with development versions of py2app.

mrclary · 2022-04-22T21:50:25Z

Okay, it looks like this is not a bug in py2app but a result of the migration to handling the distributions with importlib rather than pkg_resources. @ronaldoussoren, can you confirm this?

Working with local builds, I can launch Spyder's Python from the terminal and check entry points. Pre py2app<0.28 shows entry points for spyder.plugins from pkg_resources and importlib; py2app=0.28 shows entry points only from importlib.

py2app<0.28

>> PYTHONHOME=Spyder.app/Contents/Resources
>> Spyder.app/Contents/MacOS/python -c "import pkg_resources; print(list(pkg_resources.iter_entry_points('spyder.plugins')))"
[EntryPoint.parse('terminal = spyder_terminal.terminalplugin:TerminalPlugin'), EntryPoint.parse('appearance = spyder.plugins.appearance.plugin:Appearance'), EntryPoint.parse('application = spyder.plugins.application.plugin:Application'), EntryPoint.parse('breakpoints = spyder.plugins.breakpoints.plugin:Breakpoints'), EntryPoint.parse('completions = spyder.plugins.completion.plugin:CompletionPlugin'), EntryPoint.parse('editor = spyder.plugins.editor.plugin:Editor'), EntryPoint.parse('explorer = spyder.plugins.explorer.plugin:Explorer'), EntryPoint.parse('find_in_files = spyder.plugins.findinfiles.plugin:FindInFiles'), EntryPoint.parse('help = spyder.plugins.help.plugin:Help'), EntryPoint.parse('historylog = spyder.plugins.history.plugin:HistoryLog'), EntryPoint.parse('internal_console = spyder.plugins.console.plugin:Console'), EntryPoint.parse('ipython_console = spyder.plugins.ipythonconsole.plugin:IPythonConsole'), EntryPoint.parse('layout = spyder.plugins.layout.plugin:Layout'), EntryPoint.parse('main_interpreter = spyder.plugins.maininterpreter.plugin:MainInterpreter'), EntryPoint.parse('mainmenu = spyder.plugins.mainmenu.plugin:MainMenu'), EntryPoint.parse('onlinehelp = spyder.plugins.onlinehelp.plugin:OnlineHelp'), EntryPoint.parse('outline_explorer = spyder.plugins.outlineexplorer.plugin:OutlineExplorer'), EntryPoint.parse('plots = spyder.plugins.plots.plugin:Plots'), EntryPoint.parse('preferences = spyder.plugins.preferences.plugin:Preferences'), EntryPoint.parse('profiler = spyder.plugins.profiler.plugin:Profiler'), EntryPoint.parse('project_explorer = spyder.plugins.projects.plugin:Projects'), EntryPoint.parse('pylint = spyder.plugins.pylint.plugin:Pylint'), EntryPoint.parse('run = spyder.plugins.run.plugin:Run'), EntryPoint.parse('shortcuts = spyder.plugins.shortcuts.plugin:Shortcuts'), EntryPoint.parse('statusbar = spyder.plugins.statusbar.plugin:StatusBar'), EntryPoint.parse('toolbar = spyder.plugins.toolbar.plugin:Toolbar'), EntryPoint.parse('tours = spyder.plugins.tours.plugin:Tours'), EntryPoint.parse('variable_explorer = spyder.plugins.variableexplorer.plugin:VariableExplorer'), EntryPoint.parse('workingdir = spyder.plugins.workingdirectory.plugin:WorkingDirectory')]
>> Spyder.app/Contents/MacOS/python -c "import importlib_metadata; print(importlib_metadata.entry_points().select(group='spyder.plugins'))"
[EntryPoint(name='terminal', value='spyder_terminal.terminalplugin:TerminalPlugin', group='spyder.plugins'), EntryPoint(name='appearance', value='spyder.plugins.appearance.plugin:Appearance', group='spyder.plugins'), EntryPoint(name='application', value='spyder.plugins.application.plugin:Application', group='spyder.plugins'), EntryPoint(name='breakpoints', value='spyder.plugins.breakpoints.plugin:Breakpoints', group='spyder.plugins'), EntryPoint(name='completions', value='spyder.plugins.completion.plugin:CompletionPlugin', group='spyder.plugins'), EntryPoint(name='editor', value='spyder.plugins.editor.plugin:Editor', group='spyder.plugins'), EntryPoint(name='explorer', value='spyder.plugins.explorer.plugin:Explorer', group='spyder.plugins'), EntryPoint(name='find_in_files', value='spyder.plugins.findinfiles.plugin:FindInFiles', group='spyder.plugins'), EntryPoint(name='help', value='spyder.plugins.help.plugin:Help', group='spyder.plugins'), EntryPoint(name='historylog', value='spyder.plugins.history.plugin:HistoryLog', group='spyder.plugins'), EntryPoint(name='internal_console', value='spyder.plugins.console.plugin:Console', group='spyder.plugins'), EntryPoint(name='ipython_console', value='spyder.plugins.ipythonconsole.plugin:IPythonConsole', group='spyder.plugins'), EntryPoint(name='layout', value='spyder.plugins.layout.plugin:Layout', group='spyder.plugins'), EntryPoint(name='main_interpreter', value='spyder.plugins.maininterpreter.plugin:MainInterpreter', group='spyder.plugins'), EntryPoint(name='mainmenu', value='spyder.plugins.mainmenu.plugin:MainMenu', group='spyder.plugins'), EntryPoint(name='onlinehelp', value='spyder.plugins.onlinehelp.plugin:OnlineHelp', group='spyder.plugins'), EntryPoint(name='outline_explorer', value='spyder.plugins.outlineexplorer.plugin:OutlineExplorer', group='spyder.plugins'), EntryPoint(name='plots', value='spyder.plugins.plots.plugin:Plots', group='spyder.plugins'), EntryPoint(name='preferences', value='spyder.plugins.preferences.plugin:Preferences', group='spyder.plugins'), EntryPoint(name='profiler', value='spyder.plugins.profiler.plugin:Profiler', group='spyder.plugins'), EntryPoint(name='project_explorer', value='spyder.plugins.projects.plugin:Projects', group='spyder.plugins'), EntryPoint(name='pylint', value='spyder.plugins.pylint.plugin:Pylint', group='spyder.plugins'), EntryPoint(name='run', value='spyder.plugins.run.plugin:Run', group='spyder.plugins'), EntryPoint(name='shortcuts', value='spyder.plugins.shortcuts.plugin:Shortcuts', group='spyder.plugins'), EntryPoint(name='statusbar', value='spyder.plugins.statusbar.plugin:StatusBar', group='spyder.plugins'), EntryPoint(name='toolbar', value='spyder.plugins.toolbar.plugin:Toolbar', group='spyder.plugins'), EntryPoint(name='tours', value='spyder.plugins.tours.plugin:Tours', group='spyder.plugins'), EntryPoint(name='variable_explorer', value='spyder.plugins.variableexplorer.plugin:VariableExplorer', group='spyder.plugins'), EntryPoint(name='workingdir', value='spyder.plugins.workingdirectory.plugin:WorkingDirectory', group='spyder.plugins')]

py2app=0.28

>> PYTHONHOME=Spyder.app/Contents/Resources
>> Spyder.app/Contents/MacOS/python -c "import pkg_resources; print(list(pkg_resources.iter_entry_points('spyder.plugins')))"
[]
>> Spyder.app/Contents/MacOS/python -c "import importlib_metadata; print(importlib_metadata.entry_points().select(group='spyder.plugins'))"
[EntryPoint(name='terminal', value='spyder_terminal.terminalplugin:TerminalPlugin', group='spyder.plugins'), EntryPoint(name='appearance', value='spyder.plugins.appearance.plugin:Appearance', group='spyder.plugins'), EntryPoint(name='application', value='spyder.plugins.application.plugin:Application', group='spyder.plugins'), EntryPoint(name='breakpoints', value='spyder.plugins.breakpoints.plugin:Breakpoints', group='spyder.plugins'), EntryPoint(name='completions', value='spyder.plugins.completion.plugin:CompletionPlugin', group='spyder.plugins'), EntryPoint(name='editor', value='spyder.plugins.editor.plugin:Editor', group='spyder.plugins'), EntryPoint(name='explorer', value='spyder.plugins.explorer.plugin:Explorer', group='spyder.plugins'), EntryPoint(name='find_in_files', value='spyder.plugins.findinfiles.plugin:FindInFiles', group='spyder.plugins'), EntryPoint(name='help', value='spyder.plugins.help.plugin:Help', group='spyder.plugins'), EntryPoint(name='historylog', value='spyder.plugins.history.plugin:HistoryLog', group='spyder.plugins'), EntryPoint(name='internal_console', value='spyder.plugins.console.plugin:Console', group='spyder.plugins'), EntryPoint(name='ipython_console', value='spyder.plugins.ipythonconsole.plugin:IPythonConsole', group='spyder.plugins'), EntryPoint(name='layout', value='spyder.plugins.layout.plugin:Layout', group='spyder.plugins'), EntryPoint(name='main_interpreter', value='spyder.plugins.maininterpreter.plugin:MainInterpreter', group='spyder.plugins'), EntryPoint(name='mainmenu', value='spyder.plugins.mainmenu.plugin:MainMenu', group='spyder.plugins'), EntryPoint(name='onlinehelp', value='spyder.plugins.onlinehelp.plugin:OnlineHelp', group='spyder.plugins'), EntryPoint(name='outline_explorer', value='spyder.plugins.outlineexplorer.plugin:OutlineExplorer', group='spyder.plugins'), EntryPoint(name='plots', value='spyder.plugins.plots.plugin:Plots', group='spyder.plugins'), EntryPoint(name='preferences', value='spyder.plugins.preferences.plugin:Preferences', group='spyder.plugins'), EntryPoint(name='profiler', value='spyder.plugins.profiler.plugin:Profiler', group='spyder.plugins'), EntryPoint(name='project_explorer', value='spyder.plugins.projects.plugin:Projects', group='spyder.plugins'), EntryPoint(name='pylint', value='spyder.plugins.pylint.plugin:Pylint', group='spyder.plugins'), EntryPoint(name='run', value='spyder.plugins.run.plugin:Run', group='spyder.plugins'), EntryPoint(name='shortcuts', value='spyder.plugins.shortcuts.plugin:Shortcuts', group='spyder.plugins'), EntryPoint(name='statusbar', value='spyder.plugins.statusbar.plugin:StatusBar', group='spyder.plugins'), EntryPoint(name='toolbar', value='spyder.plugins.toolbar.plugin:Toolbar', group='spyder.plugins'), EntryPoint(name='tours', value='spyder.plugins.tours.plugin:Tours', group='spyder.plugins'), EntryPoint(name='variable_explorer', value='spyder.plugins.variableexplorer.plugin:VariableExplorer', group='spyder.plugins'), EntryPoint(name='workingdir', value='spyder.plugins.workingdirectory.plugin:WorkingDirectory', group='spyder.plugins')]

Perhaps this issue may resolved by migrating from pkg_resources to importlib for handling plugins?

It is puzzling to me, however, that Spyder starts just fine when built with py2app=0.28, but only when invoking spyder/app/start.py from a subprocess does pkg_resources fail to get entry points. The launch script Spyder.app/Contents/MacOS/Spyder should be identical to our spyder.py launch script, i.e.

from spyder.app import start
start.main()

but I cannot confirm since it is a built binary inside the app.

@ronaldoussoren, is the launch script modified by py2app in some way that would explain this behavior?

mrclary · 2022-04-22T22:03:26Z

@ccordoba12, I recommend moving forward with this PR as it resolves completely #17729. This PR is necessary, but insufficient, for fixing #17731. I will open another PR to complete the fix for that issue and move the relevant aspects of this conversation to the new PR.

ccordoba12 · 2022-04-23T00:45:03Z

As you can see, the app segfaults on exit while testing it. Don't you observe the same behavior locally?

mrclary · 2022-04-23T01:04:19Z

As you can see, the app segfaults on exit while testing it. Don't you observe the same behavior locally?

Ahh, I see. That is not a segfault; Spyder.app is working properly but the CI is capturing the ZMQ error when the IPython Console shuts down. This is issue #17615. I was curious why the CI was not capturing that error before. I suggest the following waiting on this PR until either jupyter-client is fixed or it is time to release 5.3.1, whichever comes first. If we need to release 5.3.1 before a fix for jupyter-client is ready, then we can limit jupyter-client=7.2.0 for just the macOS installer until the fix is ready.

…-ide#17615 is resolved

mrclary · 2022-04-23T01:32:46Z

@ccordoba12, it looks like the CI for the macOS installer is working with jupyter-client=7.2.0.

mrclary · 2022-04-23T01:40:16Z

Okay, hold on. The full version is failing for some reason...

ccordoba12 · 2022-04-23T03:27:25Z

I think the reason is PyQt 5.15. I mean, the app fails with that version but works well with 5.12

Edit: I just saw that the Lite app doesn't fail, so I don't know what the cause could be then.

mrclary · 2022-04-23T04:05:04Z

I think the reason is PyQt 5.15. I mean, the app fails with that version but works well with 5.12

Edit: I just saw that the Lite app doesn't fail, so I don't know what the cause could be then.

Okay, so test_app.sh starts the application and waits to see if it throws any errors; if it does so before the timeout (60s) then it will exit with non zero code and the CI will correctly identify the failure. However, if Spyder does not throw any errors, then it assumes everything is okay and attempts to shut down Spyder in a nice way. This is seen in the logs as

Timeout reached!
Handling signal: 15

Subsequent output is Spyder shutting down. However, if Spyder fails to quit after the default delay of 2s, then a hard kill is sent and the process exits with a non-zero status and the CI identifies the failure. This is seen in the logs as

Killed 9

I observed this in all the failed tests, but assumed that it was due to the ZMQ issue. However, I've been able to reproduce the issue locally by setting the delay to 1s. My hypothesis now is that the shutdown time may be a bit flakey and the delay before the hard kill is too short. Maybe PyQt 5.15 resulted in a slight increase in the shutdown time; maybe the full version takes slightly longer than the lite version to shut down; maybe the ZMQ errors also increased the shutdown time; maybe a combination of all these. In any case, the solution may be as simple as increasing the delay to hard kill.

…d 9 issue.

ccordoba12

Thanks for your help with this @mrclary!

The base branch was changed.

Fixes #17729

ccordoba12 added this to the v5.3.1 milestone Apr 22, 2022

mrclary requested a review from ccordoba12 April 22, 2022 22:00

mrclary force-pushed the issue-17729 branch from e29c0d9 to fc81084 Compare April 22, 2022 22:01

mrclary mentioned this pull request Apr 22, 2022

Spyder fails to restart after selecting Restart from the macOS standalone application menu bar #17731

Closed

mrclary added 2 commits April 22, 2022 18:13

Remove macOS app case in closing window.

57f9db9

Temporarily limit jupyter-client for the macos installer until spyder…

496a49a

…-ide#17615 is resolved

mrclary force-pushed the issue-17729 branch from fc81084 to 496a49a Compare April 23, 2022 01:13

Increase delay to hard kill to 10s; maybe this will resolve the Kille…

71d13cd

…d 9 issue.

ccordoba12 previously approved these changes Apr 23, 2022

View reviewed changes

ccordoba12 changed the base branch from master to 5.x April 23, 2022 17:00

ccordoba12 approved these changes Apr 23, 2022

View reviewed changes

ccordoba12 merged commit 1c9aca5 into spyder-ide:5.x Apr 23, 2022

ccordoba12 added a commit that referenced this pull request Apr 23, 2022

Merge from 5.x: PR #17732

158669e

Fixes #17729

ccordoba12 assigned mrclary Apr 23, 2022

mrclary deleted the issue-17729 branch April 23, 2022 17:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PR: Remove macOS app flow fork in closing Spyder #17732

PR: Remove macOS app flow fork in closing Spyder #17732

mrclary commented Apr 22, 2022 •

edited by ccordoba12

Loading

mrclary commented Apr 22, 2022

mrclary commented Apr 22, 2022

mrclary commented Apr 22, 2022

mrclary commented Apr 22, 2022

mrclary commented Apr 22, 2022

mrclary commented Apr 22, 2022

ccordoba12 commented Apr 23, 2022 •

edited

Loading

mrclary commented Apr 23, 2022

mrclary commented Apr 23, 2022

mrclary commented Apr 23, 2022

ccordoba12 commented Apr 23, 2022 •

edited

Loading

mrclary commented Apr 23, 2022

ccordoba12 left a comment

PR: Remove macOS app flow fork in closing Spyder #17732

PR: Remove macOS app flow fork in closing Spyder #17732

Conversation

mrclary commented Apr 22, 2022 • edited by ccordoba12 Loading

Description of Changes

Issue(s) Resolved

Affirmation

mrclary commented Apr 22, 2022

mrclary commented Apr 22, 2022

mrclary commented Apr 22, 2022

mrclary commented Apr 22, 2022

mrclary commented Apr 22, 2022

mrclary commented Apr 22, 2022

ccordoba12 commented Apr 23, 2022 • edited Loading

mrclary commented Apr 23, 2022

mrclary commented Apr 23, 2022

mrclary commented Apr 23, 2022

ccordoba12 commented Apr 23, 2022 • edited Loading

mrclary commented Apr 23, 2022

ccordoba12 left a comment

Choose a reason for hiding this comment

mrclary commented Apr 22, 2022 •

edited by ccordoba12

Loading

ccordoba12 commented Apr 23, 2022 •

edited

Loading

ccordoba12 commented Apr 23, 2022 •

edited

Loading