Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ghostty: add nixos tests, add build options, fix x11 backend #368726

Merged
merged 7 commits into from
Dec 29, 2024

Conversation

getchoo
Copy link
Member

@getchoo getchoo commented Dec 28, 2024

Follow up on #368404

Big changes here include:

  • Exposing all major build configuration options Seems this isn't something wanted
  • Removing some un-needed dependencies
  • Adding more automated testing
  • Fixing start up on X11

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@getchoo
Copy link
Member Author

getchoo commented Dec 28, 2024

@ofborg build ghostty ghostty.passthru.tests

@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Dec 28, 2024
@getchoo getchoo force-pushed the pkgs/ghostty/tests-and-stuff branch from bcc5ad2 to 1b713d3 Compare December 28, 2024 06:05
@arathunku
Copy link
Contributor

arathunku commented Dec 28, 2024

👋 for ghostty package, I'm getting a build error both on master and this PR, by any chance @getchoo have you run into it while making changes here? If not - that's alright, I'll probably open a seperate issue about it. I already raised this issue on Discord too but no success. In theory, it should be working without any problem.

build log (x86_64-linux)

ghostty> Using versionCheckHook
ghostty> Running phase: unpackPhase
ghostty> unpacking source archive /nix/store/glwxh7ircyd1hzksxpz8yw3a0ch41ym3-source
ghostty> source root is source
ghostty> Running phase: patchPhase
ghostty> Running phase: updateAutotoolsGnuConfigScriptsPhase
ghostty> Running phase: configurePhase
ghostty> no configure script, doing nothing
ghostty> Running phase: buildPhase
ghostty> zig build flags: -Dcpu=baseline -Doptimize=ReleaseFast --color off --system /nix/store/p9p2056gs8cqqiwsnrpdvhk12rbd40si-ghostty-cache-1.0.0 -Dversion-string=1.0.0 -Dapp-runtime=gtk -Dfont-backend=fontconfig_freetype -Dgtk-adwaita=true -Drenderer=opengl -fsys=glslang --search-prefix /nix/store/4nag25dah73pdnz3pb6nii9ly2fwvshc-glslang-15.0.0
ghostty> buildPhase completed in 1 minutes 41 seconds
ghostty> Running phase: glibPreInstallPhase
ghostty> Running phase: glibPreInstallPhase
ghostty> Running phase: glibPreInstallPhase
ghostty> Running phase: installPhase
ghostty> zig install flags: -Dcpu=baseline -Doptimize=ReleaseFast --color off --system /nix/store/p9p2056gs8cqqiwsnrpdvhk12rbd40si-ghostty-cache-1.0.0 -Dversion-string=1.0.0 -Dapp-runtime=gtk -Dfont-backend=fontconfig_freetype -Dgtk-adwaita=true -Drenderer=opengl -fsys=glslang --search-prefix /nix/store/4nag25dah73pdnz3pb6nii9ly2fwvshc-glslang-15.0.0 --prefix /nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0
ghostty> Moving /nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0/share/terminfo to /nix/store/w9xypqw2r23sranky2gglh3rrnpqpif5-ghostty-1.0.0-terminfo/share/terminfo
ghostty> Running phase: glibPreFixupPhase
ghostty> Running phase: glibPreFixupPhase
ghostty> Running phase: glibPreFixupPhase
ghostty> Running phase: gappsWrapperArgsHook
ghostty> Running phase: dropIconThemeCache
ghostty> Running phase: dropIconThemeCache
ghostty> Running phase: dropIconThemeCache
ghostty> Running phase: fixupPhase
ghostty> Moving /nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0/share/man to /nix/store/zahslrs9r7i377jhblxdrz42d83jnxa9-ghostty-1.0.0-man/share/man
ghostty> Wrapping program '/nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0/bin/ghostty'
ghostty> shrinking RPATHs of ELF executables and libraries in /nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0
ghostty> shrinking /nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0/bin/ghostty
ghostty> shrinking /nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0/bin/.ghostty-wrapped
ghostty> checking for references to /tmp/nix-shell-3025988-0/nix-build-ghostty-1.0.0.drv-0/ in /nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0...
ghostty> patching script interpreter paths in /nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0
ghostty> stripping (with command strip and flags -S -p) in  /nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0/bin
ghostty> shrinking RPATHs of ELF executables and libraries in /nix/store/zahslrs9r7i377jhblxdrz42d83jnxa9-ghostty-1.0.0-man
ghostty> checking for references to /tmp/nix-shell-3025988-0/nix-build-ghostty-1.0.0.drv-0/ in /nix/store/zahslrs9r7i377jhblxdrz42d83jnxa9-ghostty-1.0.0-man...
ghostty> gzipping man pages under /nix/store/zahslrs9r7i377jhblxdrz42d83jnxa9-ghostty-1.0.0-man/share/man/
ghostty> patching script interpreter paths in /nix/store/zahslrs9r7i377jhblxdrz42d83jnxa9-ghostty-1.0.0-man
ghostty> shrinking RPATHs of ELF executables and libraries in /nix/store/w9xypqw2r23sranky2gglh3rrnpqpif5-ghostty-1.0.0-terminfo
ghostty> checking for references to /tmp/nix-shell-3025988-0/nix-build-ghostty-1.0.0.drv-0/ in /nix/store/w9xypqw2r23sranky2gglh3rrnpqpif5-ghostty-1.0.0-terminfo...
ghostty> patching script interpreter paths in /nix/store/w9xypqw2r23sranky2gglh3rrnpqpif5-ghostty-1.0.0-terminfo
ghostty> Running phase: installCheckPhase
ghostty> Executing versionCheckPhase
ghostty> Did not find version 1.0.0 in the output of the command /nix/store/w8k53c0zkc212kazsfrlvscq0rx70wba-ghostty-1.0.0/bin/ghostty --version
ghostty> ghostty: error while loading shared libraries: libappstream.so.5: cannot open shared object file: No such file or directory
error: builder for '/nix/store/crmzr07hygrqszpqjh5yalbzkdgzhwq6-ghostty-1.0.0.drv' failed with exit code 2
error: 1 dependencies of derivation '/nix/store/v9xn9wpgvzblj37prrckrld981d34ykn-ghostty.drv' failed to build
error: 1 dependencies of derivation '/nix/store/7hx97klx0dcdh48xkn9f29ihdawhi25w-ghostty-fish-completions.drv' failed to build

@getchoo
Copy link
Member Author

getchoo commented Dec 28, 2024

👋 for ghostty package, I'm getting a build error both on master and this PR, by any chance @getchoo have you run into it while making changes here?

Can't say I have :(

An error about libappstream is actually very confusing as I'm not sure what could be using it. As this is already happening on master though, I would encourage a separate issue to investigate further

@arathunku
Copy link
Contributor

@getchoo thanks, I opened the issue here #368742

@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Dec 28, 2024
@ofborg ofborg bot requested review from pluiedev and jcollie December 28, 2024 08:15
@ofborg ofborg bot added 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 1 10.rebuild-linux: 1 labels Dec 28, 2024
# https://github.com/ghostty-org/ghostty/blob/4b4d4062dfed7b37424c7210d1230242c709e990/src/font/main.zig#L94
fontBackend ? if stdenv.hostPlatform.isDarwin then "coretext" else "fontconfig_freetype",
# https://github.com/ghostty-org/ghostty/blob/4b4d4062dfed7b37424c7210d1230242c709e990/PACKAGING.md#build-options
optimizeLevel ? "ReleaseFast",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this? It doesn't sound like any user could possibly want to use anything but ReleaseFast.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was asked for previously. I think it would also make sense to expose this directly, as the way we override zig.hook makes it difficult to change this flag directly

I've also found it useful in iterating on this expression at least :p

pkgs/by-name/gh/ghostty/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/gh/ghostty/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/gh/ghostty/package.nix Outdated Show resolved Hide resolved
This is meant to make cross platform support a bit easier. The options
are kept private as they aren't meant to be touched by end users
pkgs/by-name/gh/ghostty/package.nix Show resolved Hide resolved
pkgs/by-name/gh/ghostty/package.nix Show resolved Hide resolved
pkgs/by-name/gh/ghostty/package.nix Outdated Show resolved Hide resolved
ln -sf "$vim" "$out/share/vim/vimfiles"
mv $out/share/vim/vimfiles $vim
rmdir $out/share/vim
ln -s $vim $out/share/vim-plugins
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$vim needs to be added to $out/nix-support/propagated-user-env-packages as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? programs.vim in NixOS already add share/vim-plugins to environment.pathsToLink and using the vim output with home-manager's programs.vim.plugins will work as expected. Installing the plugin to the top-level will do nothing AFAIK

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needs to be added to $out/nix-support/propagated-user-env-packages so that it is installed when Ghostty is installed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's already installed at $out/share/vim-plugins whenever the ghostty binary is. There's no point installing the plugin itself to the a profile; that only belongs in plugin wrappers, etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just vim that needs the extra config and adding that config 'installs' the output. Neovim appears to just work.

# https://github.com/ghostty-org/ghostty/blob/4b4d4062dfed7b37424c7210d1230242c709e990/src/font/main.zig#L94
fontBackend = if stdenv.hostPlatform.isDarwin then "coretext" else "fontconfig_freetype";
# https://github.com/ghostty-org/ghostty/blob/4b4d4062dfed7b37424c7210d1230242c709e990/src/renderer.zig#L51-L52
renderer = if stdenv.hostPlatform.isDarwin then "metal" else "opengl";
in
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that letting the user modify these options really adds anything but more work for us (or the Ghostty developers) when people inevitably choose incompatible or non-functional combinations and then report bugs. Ghostty will automatically choose the right runtimes, backends, and renderers depending on the platform. These should be removed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that letting the user modify these options really adds anything but more work for us (or the Ghostty developers) when people inevitably choose incompatible or non-functional combinations and then report bugs.

While I very much disagree with this (people can already set these flags with zigBuildFlags or when compiling manually and do the same thing...plus I don't think making it easier to play around with these options will actually influence people to go enable them and then complain here or upstream when they're clearly not supported defaults) these options aren't able to be modified by the user already. Previously they were, but after conversation with @pluiedev they're now in this let ...; in block

The main goal here is to avoid dumping all of our dependencies into broad isDarwin and isLinux checks, especially when in the future more backends/platforms may be supported, which features/dependencies overlapping between

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the goal is only platform support, you should simply make the deps optional depending on platform.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said

The main goal here is to avoid dumping all of our dependencies into broad isDarwin and isLinux checks

I believe this makes future platform support much easier, as chances are BSD support for example would probably use similar, but not the exact same features and dependencies; separating these features into their own distinct booleans makes mixing and matching these relatively simple and a lot more comprehensible. This is also modeled almost exactly after how the build scripts determine whether or not the dependencies are needed, which I think is a good thing to follow

versionCheckHook,
wrapGAppsHook4,
zig_0_13,
zlib,
# https://github.com/ghostty-org/ghostty/blob/4b4d4062dfed7b37424c7210d1230242c709e990/build.zig#L106
withAdwaita ? true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also don't think that there's any value in letting the user build the package without Adwaita. It can be disabled at runtime if the user really needs to.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should also be noted that there are a number of developers that work on the GTK frontend that want to eliminate the possibility of building without Adwaita. There are many features that don't work at all without Adwaita (tab overviews is the big one) and other features have to be implemented twice, once for plain GTK, and once for GTK+Adwaita.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be useful to do if you want to minimise your dependency tree. Though you'd be hard-pressed to get rid of Adwaita on any desktop system.

Still, use-cases where one doesn't need any fancy UI elements at all exist. The first thing I did when trying out ghostty was to disable the bar.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also don't think that there's any value in letting the user build the package without Adwaita.

I think upstream may disagree with you there, at least for now. Why else would this be an option?

It should also be noted that there are a number of developers that work on the GTK frontend that want to eliminate the possibility of building without Adwaita.

If they do it'll be easy to remove. I don't see much reason in hiding this option in the meantime

pkgs/by-name/gh/ghostty/package.nix Show resolved Hide resolved
@getchoo getchoo force-pushed the pkgs/ghostty/tests-and-stuff branch from a1f5e53 to 2e5fdf1 Compare December 28, 2024 18:41
Forcing linkage isn't enough for Zig's `dlopen()` call. Let's just point
it towards the exact path instead
@getchoo getchoo force-pushed the pkgs/ghostty/tests-and-stuff branch from f6c194a to f4d4b22 Compare December 29, 2024 04:28
@getchoo
Copy link
Member Author

getchoo commented Dec 29, 2024

Hoping to get this merged soon so the x11 issues doesn't hit nixos-unstable 🤞

@getchoo getchoo added 12.approvals: 1 This PR was reviewed and approved by one reputable person 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in the package labels Dec 29, 2024
@wegank wegank added 12.approvals: 3+ This PR was reviewed and approved by three or more reputable people and removed 12.approvals: 1 This PR was reviewed and approved by one reputable person labels Dec 29, 2024
@ofborg ofborg bot requested a review from pluiedev December 29, 2024 08:42
@getchoo getchoo dismissed ambroisie’s stale review December 29, 2024 09:08

Changes were addressed

@getchoo
Copy link
Member Author

getchoo commented Dec 29, 2024

Self merging to get this fix in ASAP. Feel free to make followups for other changes

@getchoo getchoo merged commit 716c941 into NixOS:master Dec 29, 2024
46 of 47 checks passed
@getchoo getchoo deleted the pkgs/ghostty/tests-and-stuff branch December 29, 2024 09:09
# Avoid using runtime hacks to help find X11
postPatch = lib.optionalString (appRuntime == "gtk") ''
substituteInPlace src/apprt/gtk/x11.zig \
--replace-warn 'std.DynLib.open("libX11.so");' 'std.DynLib.open("${lib.getLib libX11}/lib/libX11.so");'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity: any idea why this zig lookup doesn't succeed without forcing the full path?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really sure tbh. It could be the dlopen implementation internally; usually forcing linkage at runtime via -l would be enough. I also tried using -rpath before this, but no dice

Won't be a problem once ghostty-org/ghostty#3857 gets into a stable tag though ;)

getchoo added a commit to getchoo-contrib/nixpkgs that referenced this pull request Dec 29, 2024
…68726)

* ghostty: add nixos test

* ghostty: add nixosTests.allTerminfo to passthru.tests

* ghostty: factor out dependencies

This is meant to make cross platform support a bit easier. The options
are kept private as they aren't meant to be touched by end users

* ghostty: add optimizationLevel option

* ghostty: cleanup outputs

* ghostty: fix x11 backend

Forcing linkage isn't enough for Zig's `dlopen()` call. Let's just point
it towards the exact path instead

* ghostty: add darwin to meta.platforms

(cherry picked from commit 716c941)
Mic92 pushed a commit to Mic92/nixpkgs that referenced this pull request Dec 29, 2024
…68726)

* ghostty: add nixos test

* ghostty: add nixosTests.allTerminfo to passthru.tests

* ghostty: factor out dependencies

This is meant to make cross platform support a bit easier. The options
are kept private as they aren't meant to be touched by end users

* ghostty: add optimizationLevel option

* ghostty: cleanup outputs

* ghostty: fix x11 backend

Forcing linkage isn't enough for Zig's `dlopen()` call. Let's just point
it towards the exact path instead

* ghostty: add darwin to meta.platforms
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 10.rebuild-linux: 1 11.by: package-maintainer This PR was created by the maintainer of the package it changes 12.approvals: 3+ This PR was reviewed and approved by three or more reputable people 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in the package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants