Make Spyder work with PyQt 5.15

[goanpeca]

Using this issue to track the discussion on how to approach this task. Turns out for open source there is no LTS support anymore :-(

The idea is to be able to ship a standalone Spyder application using this 5.15 (that will probably use pip packages to create the installer using, but we will still need conda packages for conda users).

People that have worked on this packaging include:

• @ccordoba12
• @mingwandroid
• @isuruf

Time estimates

A rough estimate could be about month per OS (Win, Linux, Mac), with 20/hrs per week. This will depend on the OS and changes in the Chrome compilation steps.

Notes

https://www.qt.io/blog/qt-offering-changes-2020

LTS and offline installer to become commercial-only
Starting with Qt 5.15, long term support (LTS) will only be available to commercial customers. This means open-source users will receive patch-level releases of 5.15 until the next minor release will become available. This means that we will handle Qt 5.15 in the same way as e.g. 5.13 or 5.14 for open source users.

[goanpeca]

Pinging @rgommers, @nerohmot

[isuruf]

5.15 is not an LTS and qt company doesn't provide LTS for open source anymore.

[goanpeca]

Ok! thanks @isuruf for the input! Is it worth then building 5.14 vs. 5.15 under these circumstances?

[ccordoba12]

@isuruf and @mingwandroid, we pinged you in this issue because we'd like to know if you could help us to create conda packages for Qt 5.15 through a Community Work Order from Quansight (so, basically being paid to do this job) in the following months.

To better approach this monstrous project, I was thinking we could split the job among the three of us, like this:

• @ccordoba12: macOS
• @isuruf: Linux
• @mingwandroid: Windows

We could also help each other to test the packages each one has created and if one finishes first, he could help the others. That's how Ray and me worked on this when I was part of Anaconda, so I think it could work well for this project too.

What do you think?

[nerohmot]

I wasn't thinking too specific when I said "LTS".
Maybe there is no Long Time Support from the qt company, but I understood that there is likely not to be a 5.16 but rather a 6.0 ...
If so, 5.15 is a "milestone", and will be around for a long time, without new things added ... that makes it Long Time Stable 😁
People could use it 'for a long time' IF the 3 OS-es implement the same stuff. What I am driving at is that for example in 5.12 Windows has the QAudioRecorder, where as Linux in any case has not (or is raising an exception).
It would be desirable that 5.15 would implement the same things for all 3 OS-es.
Spyder could then use the 5.15 cross platforms for a long time and focus on other stuff instead of "battling with the complex qt stuff"

[mingwandroid]

Why would updating to 5.15 be such a big job?

[stonebig]

In waiting for this to happen, would it be possible to relax the constraint "pyqt5<5.13", for pip users ? (<5.16 is my dream)

[mingwandroid]

It's quite a big job but I'm working on it. Not for the next few days though, I'm celebrating... Anyone is welcome to help out. The work is happening on AnacondaRecipes qt-feedstock in the master branch. A great deal of things build. Windows is nearly complete, for example and webengine compiles to a large extent on other platforms.

[stonebig]

any hope ?

[ccordoba12]

Sorry, we didn't have time to do it for 4.2.0. Hopefully for 4.2.1, to be released in two months or so.

[jessie828]

Anyone is welcome to help out.

Is there anything concrete you need help with?

[ccordoba12]

Hey @jessie828, thanks for asking. On the Spyder side, we need to make our test suite pass with PyQt 5.15 so we can support that version.

We'd also love to give the Anaconda team to create packages for Qt/PyQt 5.15, but we're swamped at the moment.

[ccordoba12]

We will update this requirement for our 5.2.0 version, to be released next month.

[Kochise]

I'm wondering, pip is like 9MB on hard drive yet https://pypi.org/project/PyQt5/ is at 5.15.4 (released in March 2021).

On the other hand, Anaconda is like 3GB once installed but https://anaconda.org/anaconda/pyqt is stuck at 5.9.2 (released in November 2019, regarding pip history).

Launching my Debian 10 VM, all I get is Spyder 3.3.3 from Synaptic (released in February 2019, regarding https://github.com/spyder-ide/spyder/tags?after=v3.3.4). Python3 is stuck at 3.7.3 (released in March 2019, regarding https://www.python.org/downloads/release/python-373/).

Why using "official" overweight channels like Conda or Synaptic gets you 2 years old obsolete packages, while using more lightweight manager like pip gets you relatively up-to-date packages ?

Why so many concurrent ways to install packages with so many different outcomes ?

I mean, on Windows or Mac, download the most recent setup/installer from https://github.com/spyder-ide/spyder/releases/, run it and voilà, done. Why everything cannot be that simple ? Not even speaking about the mess regarding virtualenv that duplicate binaries and leaves trails all around...

[texadactyl]

@Kochise

If you are a miniconda or anaconda user, install spyder with the conda-forge channel (conda -c parameter). That is normally only 1 or 2 days behind pypi (close enough).

What is "official" is in the mind of the beholder. Open source is like 1000 unpaid flowers blooming.

[Kochise]

@texadactyl Yeah, I get that, but "official" channels lagging not 2 days but 2 years behind if kinda weird in the tech industry. To say the least.

[ccordoba12]

Debían pavakges are always (very) outdated. That's like it's signature trademark.

Anaconda's PyQt packages are outdated because they are really hard to compile and the person in charge of that sadly passed away some months ago.

Our installers are not the final solution because we need to rely on Anaconda/conda-forge to install packages that don't come with them. Perhaps the best solution would be to have a package and environment manager included in our installers, so that people don't have to deal with this mess by themselves. The problem is that is hard to get it right.

[Kochise]

@ccordoba12 Indeed, but installing Spyder using pip works pretty well, so why limiting yourself to the outdated packages from Conda ? Just state pip becoming your official installation source that can even be performed from up-to-date Python3 distributions, say 3.9.7 (released late August 2021, regarding https://www.python.org/downloads/release/python-397/). Hence no more Python3 3.7.3 from 2019.

Just check the packages you need are available on https://pypi.org/ and that should do it. No more Conda dependencies of obsolete packages \o/

Btw, sorry to hear about the loss of the PyQt packages' maintainer. I know about the mess of compiling Qt first hand, and their API breakage also being their "signature trademark".

[Kochise]

Just to make things clear : abandon Conda, it's outdated and obsolete

https://anaconda.org/anaconda/pyqt : 5.9.2
https://pypi.org/project/PyQt5/ : 5.15.6

Plus Conda is huuuuuge.

[stonebig]

A huuuuuge user base too

[Kochise]

Suuuure, but I'd like to live in 2021, not 2019.

[j9ac9k]

I'm not sure what features of Qt 5.10+ Spyder would use that are not available in 5.9. Don't get me wrong, in 5.14+ there is some better handling for hidpi monitors on Windows; but I'm not sure what functionality is available in 5.15 that Spyder would be able to use.

Does the conda version of PyQt5 5.9 work on Python 3.9+?

[ccordoba12]

but I'm not sure what functionality is available in 5.15 that Spyder would be able to use.

We will probably use the ability to send files to the trash can when removing them from Files. But we'll still keep supporting older PyQt versions for Anaconda.

Does the conda version of PyQt5 5.9 work on Python 3.9+?

Yep, it does.

[Kochise]

@j9ac9k For instance, Enaml's "live editor" requires QScintilla that is not compatible with Spyder's PyQt5 5.9.

I originally raised the problem in #15986

One can always create as many "virtualenv" as needed, but why not simply upgrade to latest version ? Are people still working with PyQt4 as well ? Python 3.5 ?

[j9ac9k]

@j9ac9k For instance, Enaml's "live editor" requires QScintilla that is not compatible with Spyder's PyQt5 5.9.

I originally raised the problem in #15986

Oh right, hadn't thought of QScintilla, good point 👍🏻

One can always create as many "virtualenv" as needed, but why not simply upgrade to latest version ? Are people still working with PyQt4 as well ? Python 3.5 ?

If someone is still installing PyQt4 on a newly created virtualenv I would love to know how they're doing it in a way to be reliable cross-platform. This was one of the issues that forced me to drop Qt4 support, configuring CI platforms to use it was getting to be a constant source of head aches.

[Kochise]

@j9ac9k Indeed, at least Erlang solves this more gracefully by allowing the installation of different versions of a same "package", the "application" referring to a specific version in a "requirements.txt"-like way.

https://www.erlang.org/doc/man/app.html
https://learnyousomeerlang.com/release-is-the-word
https://stackoverflow.com/questions/27878718/how-to-version-an-erlang-project

Hence you don't even have to worry yourself with "virtualenv", just pick the right package version without conflict on your disk. It is quite simply handled this way :

https://www.erlang.org/doc/system_principles/versions.html

[ccordoba12]

Sorry to disappoint you guys but I'm going to move this for 5.2.1. Unfortunately we're getting a lot of segfaults with PyQt 5.15 (specially when Spyder is closing) and solving them is very difficult and time consuming.

[j9ac9k]

@ccordoba12 Is there an issue tracking these segfaults?

I recently ran into segfaults on close when I migrated my own tool to PyQt5 5.15. The issue with my system had to do with I weas emitting logging.LogRecord messages via signal/slot mechanism to a slot which was no longer present during shut down. My fix was to manually disconnect that signal on close. No clue if you're running into anything similar.

[ccordoba12]

Is there an issue tracking these segfaults?

No, I'm doing fixes directly in PR #16409. Some seem related to the way we now handle teardown of plugins (see PR #16012) and others to hanging threads (i.e. threads that are running while the window is closing).

[rear1019]

Sorry to disappoint you guys but I'm going to move this for 5.2.1. Unfortunately we're getting a lot of segfaults with PyQt 5.15 (specially when Spyder is closing) and solving them is very difficult and time consuming.

@ccordoba12 This might be related: We experienced a reproducible SEGFAULT whenever Spyder is closed after switching to v5.1.5. We were able to reproduce the issue with the following systems:

• Ubuntu 20.04 LTS with Qt 5.15.0 (+ a few patches if I remember correctly) and PySide2 5.15.0
• Arch Linux with with Qt 5.15.2 and PyQT 5.15.4 (most recent versions as of October; Arch packages Qt with KDE patches)

Window 10 (no idea which release) seems to be not affected.

The backtrace is basically the same in both cases (except for PySide/PyQt differences) and allowed us to pin-down the issue. The culprit is the the setStyle() call in ApplicationToolbar and MainWidgetToolbar (see below). Removing the setStyle() calls fixed the SEGFAULT. This is probably a Qt issue and not related to Python bindings [1].

Note: QThread/QProcess warnings (see e.g. #16585) conceal the SEGFAULT (unless gdb is used).

[1] https://bugreports.qt.io/browse/PYSIDE-1504

spyder/spyder/api/widgets/toolbars.py

Lines 265 to 267 in 28539bf

	self._style = ToolbarStyle(None)
	self._style.TYPE = 'Application'
	self.setStyle(self._style)

spyder/spyder/api/widgets/toolbars.py

Lines 297 to 299 in 28539bf

	self._style = ToolbarStyle(None)
	self._style.TYPE = 'MainWidget'
	self.setStyle(self._style)

[ccordoba12]

Thanks for the help @rear1019! I'll try to unset that style on closing then.

[kinglionsz]

PyQt5 5.15.7
PyQt5-Qt5 5.15.2
PyQt5-sip 12.11.0
PyQtWebEngine 5.15.6

Spyder doesn't start - no module named 'PyQt5.QtWebEngineWidgets'

Spyder 5.3.3 | Python 3.10.8 64-bit | Linux 5.15.68.1-microsoft-standard-WSL2
@ccordoba12