qt5: 5.12.7 -> 5.14.2

[ghost]

Motivation for this change

Fixes #74355
Fixes #81894

To-Do list:

• Fix loading of shared libraries as observed in Plasma and Quassel Core (fixed by @DIzFer, thanks!)
• Update the following broken packages or move them back to Qt 5.12 if no fix is available
  • boomerang
  • chatterino2
  • deepin.* (does not build on master)
  • diffpdf
  • digikam
  • falkon
  • golden-cheetah
  • ibus-engines.mozc
  • lumina.lumina-pdf
  • megasync
  • lxqt.*
  • mpc-qt
  • netease-cloud-music (does not start on master)
  • nheko
  • notepadqq
  • qgroundcontrol
  • qlcplus
  • qtox
  • quassel
  • quaternion
  • rockbox_utility
  • rssguard
  • sdrangel
  • smplayer
  • smtube

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[ghost]

qt5.qttools fails to build, because:

Error copying /build/qttools-everywhere-src-5.14.2/lib/cmake/Qt5AttributionsScannerTools/Qt5AttributionsScannerToolsConfigVersion.cmake to /nix/store/v52ixjwqg4dqa65qcqfrhd9b9fp8za3a-qttools-5.14.2/lib/cmake/Qt5AttributionsScannerTools/Qt5AttributionsScannerToolsConfigVersion.cmake: Cannot open /build/qttools-everywhere-src-5.14.2/lib/cmake/Qt5AttributionsScannerTools/Qt5AttributionsScannerToolsConfigVersion.cmake for input

But some desktop applications like tdesktop and qutebrowser are working fine.

[cole-h]

Diff LGTM, except for minor nits. No expert in this domain, so I'll defer to someone else (no way will my computer handle building and verifying the ~6000 affected packages lol).

[ghost]

I will run nix-review at some point.
And good news: I was able to fix the qttools build! Will try to rebuild my system with this now.

[ghost]

Now using this on my everyday system \o/
I had to override quassel src to master because it has some fixes to build with the new Qt. Otherwise it built fine after I fixed the QtAttributionsScanner problem in qttools.

[ghost]

I've been working on the Plasma 5 NixOS test. I had to update KDE frameworks, plasma5, quassel and grandlee so far to make them build with the new Qt.

Update: Plasma builds and starts fine now, but some of the patches will have to be re-done properly since wallpapers and some shared libraries are not detected properly. Take a look at https://github.com/petabyteboy/nixpkgs/commits/feature/plasma-5-18

[ghost]

I am unsure what is the best approach to get this new Qt version to NixOS users.

One possible approach is to set the new version as default from the beginning and try to fix as much as possible rightaway. This will probably be the quickest way to get as many applications as possible on the new Qt version, which is by the way required for some of the fixes to work (wayland clipboard will only be better in the applications that are built with the new Qt).
The downside of this will be the enourmous diff size and time required to review it. It might also lead to more user issues in the short-term, because I can't possibly test the functionality of all applications that use Qt.

Another approach would be to leave 5.12 as the default version and gradually move packages to the new version. This has the advantage that we can check every package thoroughly, but we risk that many applications will still use 5.12 in NixOS 20.09.

[ttuegel]

@petabyteboy In the past, we have done Qt 5 upgrades by changing the default version (the qt5 attribute) and downgrading the packages that break. We will ping the maintainers of any downgraded packages, too. This is much less work than fixing all the packages, but gives a better result than trying to gradually upgrade (which won't happen). When Qt 5.12 is eventually EOL'd upstream, we will remove it and mark any packages still using it as broken.

[DIzFer]

Quassel situation isn't very nice: both core and client seem to work when updated to latest git commit, but that's technically not a stable release, so probably wiser to kick it back to Qt5.12 until next release? Not sure how we would be supposed to remember to pull it back to the future once it's officially updated...

Other than that, I'm here because of qutebrowser, but something about QtWebEngine appears not to agree with my machine, so that I can't test. My desktop is at the moment rebuilding without QtWebEngine, which means I'll accidentally test some of #84542 too.

[ghost]

Hey, let me see what I can do for you in terms of QtWebEngine and qutebrowser. I will set up a small repository that includes an overlay and a fixed version of nixpkgs, similar to colemickens' nixpkgs-chromium and push the build results to a Cachix cache.

About quassel: The daemon has problems finding it's database library, so even then it doesn't work. I'm just using the client with the latest master commit, but that shouldn't be something we commit to nixpkgs. Instead we should either backport the necessary patches to the last Quassel release or leave Quassel with Qt5.12 until there is a new release.

[DIzFer]

That would be most gracious of you, but nevermind, I've somehow successfully built qutebrowser and webengine without realising... As you can tell from my comment, nix frequently conspires against my expectations. Which in the end means this seems to build for me all right!

Besides a few other apps you don't mention to be broken not being broken, I checked on some others that might affect me:

• nheko is supposedly fixed in nheko: 0.6.4 -> 0.7.1 #85922 (merged). Not been able to verify yet, since I'm apparently terrible at github-git translation.
• quaternion gets an update in quaternion: update, libquotient: init #84160 (merged) which lets it both build and run.
• digikam fails way before any real problem, an apparent checksum mismatch on 97ce35e#diff-c7c1cea8cd1fc212a3af4b8c39e28e87R47. Apparently we're six months behind, which may be due to KDE not syncing their upstream repo to the github mirror we download from. Unsure on how to proceed in this case (Update from upstream directly? Find whoever's responsible for KDE's mirrors on github so they can check what's up and wait until that's cleared up?).

I'll report with more details about digikam once I finish shaving the required yaks, which don't appear to be few...

[DIzFer]

Apparently a big enough yak and many yaks in a row look very similar. Digikam failed because building libqtav fails. libqtav has a new version github is displaying slightly different, apparently tags != releases? Whatever, but maybe that's why we haven't updated it automatically. In any case, latest libqtav doesn't build either, but HEAD felt like working. Currently 60% along building qtwebkit (dear god why help me not another html engine), and only one more derivation left before digikam's turn.

Additionally, reading through the list, rssguard caught my attention, I read up on it, and out of curiosity I decided to attempt to fix it. Once again, it's one release behind, but latest stable doesn't build either. Besides qmake being bollocks, I don't understand the error in question (make[1]: *** No rule to make target '../../localization/qtbase_uk.qm', needed by 'rcc/qrc_rssguard.o'. Stop.), so I'm stopping here with this one.

[ghost]

Thanks for helping with testing. I have rebased on top of master and confirmed the ones you mentioned to be working.

Unfortunately I ran into another issue that is giving me headaches:
When I first tested this Qt update I wasn't aware of the seperate update schedules for Plasma, kde-frameworks and Qt. Because of this and a build failure with Qt 5.14.2, kdeFrameworks 5.56 and Plasma 5.17.5, I tested the new Plasma 5.18.x and Qt 5.14.x in one branch. The Plasma settings application and some other things were not properly working on this branch, but back then I thought this issue was caused by the Plasma update, which it is not. The new Plasma with the old Qt works fine, while the old Plasma with the new Qt also has the issue.
TL;DR: Something with Qt applications loading shared libraries is seriously broken. I have observed this in the Plasma settings application and in Quassel core not being able to find its Postgres database adapter library.

If anyone is in need of prebuilt QtWebEngine binaries for Qt5.14, maybe because they want to use a QtWebEngine-based browser with TLSv1.3 support, this is available here.

I can also confirm the following applications still don't build for me after the rebase:
chatterino2 boomerang digikam quassel diffpdf falkon rssguard

[DIzFer]

Right, the plasma issues make sense. You just prevented me from flooding the plasma update PR with panic and "everything is broken" comments :'D

Short version is, yeah, randomly missing QML modules, no wallpapers, most systemsettings KCMs are broken.

I just contacted the email listed at the github.com/kde organisation, in case that reaches someone that can fix digikam being out of date.

So you go to all this effort to get a more secure QtWebEngine... and I just wanted to have a browser not blind me in every website. Nice priorities, go me :P

[DIzFer]

I have news about digikam: It failed to build! Hooray!

When trying to re-build it with useful logs (nix log appears to be unhelpful this time, possibly because it failed before reaching buildPhase?), well, I was digging out how to update kde-frameworks, so they were now pointing to 5.69 (unstable has 5.68)... so I'm rebuilding the universe once again. [1/95/112] ATM, apparently everything remaining needs kio to be done.

While I'm at it, I'll see if the rest of plasma gets any better with new frameworks. Applications seems to be up-to-date.

[DIzFer]

Something with Qt applications loading shared libraries is seriously broken. I have observed this in the Plasma settings application and in Quassel core not being able to find its Postgres database adapter library.

Wait, are Qt applications failing in general? I've only noticed brokenness with KDE stuffs, which would point to "updating frameworks" being a helpful step. But you're right, quassel doesn't seem to depend on that. Welp :(

[ghost]

If you experience the "everything is broken" with the current state of the Plasma update PR, without the Qt update, please flood it with comments. I would expect most these problems to only occur when the Qt update is applied as well.

Multiple failures I have anlayzed were caused by an issue with the outputs of qttools. For example the mpc-qt build fails, because qttools-5.14.2-bin/bin/lupdate is called, but this does not exist. diffpdf fails becuase it tries to call qttools-5.14.2-bin/bin/lrelease, but this does not exist. Both binaries exist in the dev output and did not exist in the bin output of qttools in 5.12.

About the general problems with some applications built agains new Qt: Quassel does actually use quite a few of the k* libraries. I'm not sure if the problem lies in the kdeFrameworks, I haven't seen an application that is broken and doesn't use the k* libraries. I guess updating the frameworks would be worth a try, but I wouldn't count on it fixing the issue.

[DIzFer]

Progress with digikam: cmake failure, with viable logs. Seems like multiple outputs are messing up cmake, and the .cmake in exiv2-dev points to a file found in main exiv2. I know this must be easily fixable, but not how exactly. I'm parking this for now.

Only editing to mention the possibility of that being exactly this issue :/

If you experience the "everything is broken" with the current state of the Plasma update PR, without the Qt update, please flood it with comments. I would expect most these problems to only occur when the Qt update is applied as well.

Yeah, between my tendency to try to do a hundred things at once and the resulting confusion, I'd expect at least some of my issues to be my own fault. I'll try to separate, but at some point these are going to have to learn to live together :P

For now, I confirm Qt 5.14 + Plasma 5.18 + Frameworks 5.69 fails miserably :(

Currently rebuilding Plasma 5.18 without Qt or Frameworks updates.

---

nix show-derivation nixpkgs.quasselDaemon doesn't seem to list anything from frameworks. The client does, though, so I'd expect that to be the reason frameworks are listed in the package .nix.

Multiple failures I have anlayzed were caused by an issue with the outputs of qttools. For example the mpc-qt build fails, because qttools-5.14.2-bin/bin/lupdate is called, but this does not exist. diffpdf fails becuase it tries to call qttools-5.14.2-bin/bin/lrelease, but this does not exist. Both binaries exist in the dev output and did not exist in the bin output of qttools in 5.12.

The split in qttools looks fun. -bin has lupdate-pro, but lupdate lives in -dev. -dev also has a lupdate-pro, but it actually is a symlink to -bin. Many of the files in qttools-dev/bin are like that, a symlink to -bin/bin. Hopefully this make more sense if one understands what qttools is, but I don't.

[reinhardt]

@Reinhart did you build from this branch without any rebase or change? So commit 8fdd19f?

Yup, exactly. That was the first error I got, before touching anything. Later I fiddled with the versions a bit and got it to build but had trouble with ssl at runtime.

[ghost]

Unless anyone raises new or existing concerns I will merge this today

[jonringer]

nice work everyone :)

[vcunat]

Can some of you have a look at the channel-blocking darwin failure? #95199