Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use vcpkg for Linux CI #7316

Open
wants to merge 69 commits into
base: master
Choose a base branch
from
Open

Use vcpkg for Linux CI #7316

wants to merge 69 commits into from

Conversation

messmerd
Copy link
Member

@messmerd messmerd commented Jun 13, 2024

This PR moves the Linux CI builds away from our custom Ubuntu 20.04 images to a plain Ubuntu 20.04 runner with vcpkg.

  • Now using vcpkg for multiple Linux dependencies
    • Important dependencies can now stay up-to-date rather than several years old
    • There were issues with a few vcpkg packages, so I had to use system packages (for ALSA, JACK, portaudio, fltk, libgig, lv2, lilv, ...) or other solutions (for Qt) instead. Vcpkg could not build Qt without the build runner running out of memory, but for the other dependencies, the problem was that CMake could not find them or they caused linker errors.
  • Removes dependence on custom Docker images for Linux CI build
    • Uses GitHub's default Ubuntu 22.04 runner
    • Provides deps-ubuntu-22.04-gcc.txt as a living document of (almost) all required system dependencies
    • Makes getting started with LMMS development easier for new devs
  • Updates multiple dependencies including:
    • Qt: 5.12.8 --> 5.15.2 (all CI builds use 5.15.x now)
    • libsndfile: 1.0.28 --> 1.2.2 (enables opus and mp3 support)
    • fluidsynth: 2.1.1 --> 2.3.5
    • libsamplerate: 0.1.9 --> 0.2.2
    • ...
  • Updates qt5-x11embed submodule to fix deprecation warning which was treated as an error (Avoid deprecated QFlags constructor lukas-w/qt5-x11embed#6)
  • Fixed unused variable warnings in SampleDecoder.cpp which were treated as errors

I will try to do MinGW builds with vcpkg next.

@messmerd messmerd added needs code review A functional code review is currently required for this PR CI Issues/pull requests regarding continuous integration(CI) system labels Nov 20, 2024
@Rossmaxx
Copy link
Contributor

There were issues with a few vcpkg packages, so I had to use system packages (for ALSA, JACK, portaudio, fltk, libgig, lv2, lilv, ...) or other solutions (for Qt) instead.

Can't we just use qt from apt for now?

@messmerd
Copy link
Member Author

No, Ubuntu 20.04 only has Qt 5.12 available through apt, so I'm using a 3rd party tool to install Qt 5.15. It's the same tool we use for the MSVC builds.

@Rossmaxx
Copy link
Contributor

Ok i understand now. I believe using system qt might reduce steps. 22.04 (or 24) should contain the version anyway so I thought it would be better to defer qt upgrade to the system upgrade for now.

@messmerd
Copy link
Member Author

There's no real reason to put it off when it can be done now, especially since Linux is the only platform that isn't on Qt 5.15 yet.

@Rossmaxx
Copy link
Contributor

Okk then

@Rossmaxx
Copy link
Contributor

Fixed unused variable warnings in SampleDecoder.cpp which were treated as errors

Would it makes sense to fix this properly?

@messmerd
Copy link
Member Author

The unused variable warnings were for static variables defined in the vorbis header, and defining OV_EXCLUDE_STATIC_CALLBACKS is how you prevent that header from defining them.

@Rossmaxx
Copy link
Contributor

I would like to test the working before approval. Might take some days as I'm busy with exams.

@messmerd
Copy link
Member Author

messmerd commented Feb 2, 2025

Currently experiencing this linker error on the MSVC builds:

[423/1461] Linking C shared module plugins\ladspa\dc_remove_1207.dll
[424/1461] Generating D:/a/lmms/lmms/plugins/LadspaEffect/swh/ladspa/flanger_1191.c
[425/1461] Linking CXX executable lmms.exe
FAILED: lmms.exe src/lmms.lib 
C:\Windows\system32\cmd.exe /C "cd . && C:/ProgramData/chocolatey/bin/ccache.exe "C:\Program Files\CMake\bin\cmake.exe" -E vs_link_exe --msvc-ver=1929 --intdir=src\CMakeFiles\lmms.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\mt.exe --manifests  -- C:\PROGRA~2\MICROS~2\2019\ENTERP~1\VC\Tools\MSVC\1429~1.301\bin\Hostx64\x64\link.exe /nologo @CMakeFiles\lmms.rsp  /out:lmms.exe /implib:src\lmms.lib /pdb:lmms.pdb /version:0.0 /machine:x64 /debug /INCREMENTAL /subsystem:windows && C:\Windows\system32\cmd.exe /C "cd /D D:\a\lmms\lmms\build\src && "C:\Program Files\PowerShell\7\pwsh.exe" -noprofile -executionpolicy Bypass -file C:/vcpkg/scripts/buildsystems/msbuild/applocal.ps1 -targetBinary D:/a/lmms/lmms/build/lmms.exe -installedDir D:/a/lmms/lmms/build/vcpkg_installed/x64-windows/bin -OutVariable out""
LINK Pass 1: command "C:\PROGRA~2\MICROS~2\2019\ENTERP~1\VC\Tools\MSVC\1429~1.301\bin\Hostx64\x64\link.exe /nologo @CMakeFiles\lmms.rsp /out:lmms.exe /implib:src\lmms.lib /pdb:lmms.pdb /version:0.0 /machine:x64 /debug /INCREMENTAL /subsystem:windows /MANIFEST /MANIFESTFILE:src\CMakeFiles\lmms.dir/intermediate.manifest src\CMakeFiles\lmms.dir/manifest.res" failed (exit code 1104) with the following output:
LINK : fatal error LNK1104: cannot open file 'lilv-0.lib'

I wonder if it would happen on master too if it ever had a cache miss and had to rebuild the vcpkg packages.

@Rossmaxx
Copy link
Contributor

Rossmaxx commented Feb 2, 2025

I think it's unrelated. Also since you encountered it, I wanna tell you that MSVC error messages are a PITA to deal with, with not enough information to fix it. If this is indeed related, i would suggest you gpt it out.

@tresf
Copy link
Member

tresf commented Feb 3, 2025

MSVC error messages are a PITA to deal with, with not enough information to fix it. If this is indeed related, i would suggest you gpt it out.

Because they stash lines longer than 127 characters into .rsp files.... ChatGPT won't be able to help with that.

I wonder if it would happen on master too if it ever had a cache miss and had to rebuild the vcpkg packages.

Yeah I'm confused by the CI logs... is it a bad cache that it's trying to build with?

@messmerd
Copy link
Member Author

messmerd commented Feb 4, 2025

@tresf

is it a bad cache that it's trying to build with?

No, a while back I think I deleted the MSVC vcpkg cache for my fork (called vcpkg-x64-*) and now it has to rebuild the packages but fails with the lilv issue on MSVC. Even my master branch fails to build so it's not just this branch.

The error may be related to this upstream update for lilv, lv2, sord, and serd back in December: microsoft/vcpkg#42537

I'm pretty sure if LMMS/lmms's vcpkg cache were to be deleted, the same build errors would occur over there.

@messmerd
Copy link
Member Author

messmerd commented Feb 4, 2025

Unlike most of our dependencies, lilv uses Meson and doesn't provide a CMake target. We are using the .pc pkg-config file to locate it and link it.

So I think I'll try creating a FindLilv.cmake file to give it a CMake target and go from there.

@messmerd messmerd mentioned this pull request Feb 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Issues/pull requests regarding continuous integration(CI) system in progress linux needs code review A functional code review is currently required for this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants