"nixos-rebuild build-vm-with-booloader" with systemd-boot is broken again #107255

bjornfor · 2020-12-20T13:14:35Z

Describe the bug
nixos-rebuild build-vm-with-booloader on a NixOS configuration with systemd-boot as bootloader (the default for NixOS installs on UEFI systems) fails with this error:

insmod: ERROR: could not insert module /nix/store/1ibmgfr13r8b6xyn4f0wj115819f359c-linux-5.4.83/lib/modules/5.4.83/kernel/fs/efivarfs/efivarfs.ko.xz: No such device
mount: /sys/firmware/efi/efivars: mount point does not exist.

I checked that the efivarfs.ko.xz file exists, so what's missing in the kernel since it reports "No such device"?

Steps to reproduce

$ cat >configuration.nix << EOF
{
  fileSystems."/".device = "/dev/sda1";
  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;
}
EOF

$ nixos-rebuild build-vm-with-bootloader -I nixos-config=$PWD/configuration.nix -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.09.tar.gz
building Nix...
building the system configuration...
these derivations will be built:
  /nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv
  /nix/store/7dkqw00frhzfqjzn4w7fnwirldhs48bl-run-nixos-vm.drv
  /nix/store/yaw9kch9adk6xi1qzki98jcricxyz9l8-nixos-vm.drv
building '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' on 'ssh://nix-remote-build@bforsman.name'...
Formatting '/nix/store/1bzy8vqdv2bmib7qi658anc9kf7w7b6f-nixos-boot-disk/disk.img', fmt=qcow2 cluster_size=65536 compression_type=zlib size=62914560 lazy_refcounts=off refcount_bits=16
cSeaBIOS (version rel-1.13.0-48-gd9c812dda519-prebuilt.qemu.org)

iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+1FF8EEF0+1FEEEEF0 CA00

Booting from ROM...
loading kernel modules...isable)... ock
mounting Nix store...
mounting host's temporary directory...
starting stage 2 (/nix/store/rq2ishkgykpx4szwk5w8zcjk1jbp8svx-vm-run-stage2)
Creating new GPT entries in memory.
Setting name!
partNum is 0
Setting name!
partNum is 1
The operation has completed successfully.
mkfs.fat 4.1 (2017-01-24)
insmod: ERROR: could not insert module /nix/store/1ibmgfr13r8b6xyn4f0wj115819f359c-linux-5.4.83/lib/modules/5.4.83/kernel/fs/efivarfs/efivarfs.ko.xz: No such device
mount: /sys/firmware/efi/efivars: mount point does not exist.
[    1.908328] reboot: Power down
builder for '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' failed with exit code 32
error: build of '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' on 'ssh://nix-remote-build@bforsman.name' failed: builder for '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' failed with exit code 32
builder for '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' failed with exit code 1
cannot build derivation '/nix/store/7dkqw00frhzfqjzn4w7fnwirldhs48bl-run-nixos-vm.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/yaw9kch9adk6xi1qzki98jcricxyz9l8-nixos-vm.drv': 1 dependencies couldn't be built
error: build of '/nix/store/yaw9kch9adk6xi1qzki98jcricxyz9l8-nixos-vm.drv' failed

This happens with nixos-20.09 and nixos-unstable, but not nixos-20.03. (Just change the nixos-rebuild command above.)

Expected behavior
The build succeeds, like it did before (on nixos-20.03).

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Notify maintainers

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

The text was updated successfully, but these errors were encountered:

bjornfor · 2020-12-20T13:17:43Z

Disabling ~~systemd-boot~~ efi settings from configuration.nix results in

[...]
mkfs.fat 4.1 (2017-01-24)
insmod: ERROR: could not insert module /nix/store/1ibmgfr13r8b6xyn4f0wj115819f359c-linux-5.4.83/lib/modules/5.4.83/kernel/fs/efivarfs/efivarfs.ko.xz: No such device
updating GRUB 2 menu...
[...]

So the insmod error is the same, but now the init script doesn't die on mount: /sys/firmware/efi/efivars: mount point does not exist.

bjornfor · 2020-12-20T13:21:21Z

Ah, the default value of boot.loader.efi.canTouchEfiVariables is false, and with that value it actually succeeds. So it's not really the systemd-boot part of the config that is broken.

bjornfor · 2020-12-20T13:31:10Z

Oh, this comment recommends NOT to set boot.loader.efi.canTouchEfiVariables = true, because it can wear out the NVRAM: NixOS/nixos-hardware#134 (comment)

bjornfor · 2020-12-20T13:34:32Z

Ah, the default value of boot.loader.efi.canTouchEfiVariables is false, [...]

Well, except that nixos-generate-config sets it to true, so it is enabled by default for NixOS EFI installs.

bjornfor · 2020-12-20T14:42:01Z

Interestingly, the nixos systemd-boot test uses virtualisation.useEFIBoot = true. That results in success!

bjornfor · 2020-12-20T14:58:50Z

Now the question is, where to set virtualisation.useEFIBoot = true (or mkDefault config.boot.loader.systemd-boot.enable)?

I see at least these places where conditionally adding it could make sense:

Call site:

nixpkgs/nixos/default.nix

Lines 19 to 26 in 647cc06

    
           vmWithBootLoaderConfig = (import ./lib/eval-config.nix { 
        
             inherit system; 
        
             modules = 
        
               [ configuration 
        
                 ./modules/virtualisation/qemu-vm.nix 
        
                 { virtualisation.useBootLoader = true; } 
        
               ]; 
        
           }).config;

Option declaration, in qemu-vm.nix:

nixpkgs/nixos/modules/virtualisation/qemu-vm.nix

Lines 485 to 487 in 647cc06

virtualisation.useEFIBoot =

mkOption {

default = false;
Implementation, in qemu-vm.nix:

nixpkgs/nixos/modules/virtualisation/qemu-vm.nix

Lines 597 to 598 in 647cc06

virtualisation.bootDevice = mkDefault (driveDeviceName 1);

In the systemd-boot.nix module:

nixpkgs/nixos/modules/system/boot/loader/systemd-boot/systemd-boot.nix

Lines 129 to 131 in 647cc06

    
           boot.loader.grub.enable = mkDefault false; 
        
           boot.loader.supportsInitrdSecrets = true;

`nixos-rebuild build-vm-with-bootloader` currently fails with the default NixOS EFI configuration: $ cat >configuration.nix <<EOF { fileSystems."/".device = "/dev/sda1"; boot.loader.systemd-boot.enable = true; boot.loader.efi.canTouchEfiVariables = true; } EOF $ nixos-rebuild build-vm-with-bootloader -I nixos-config=$PWD/configuration.nix -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.09.tar.gz [...] insmod: ERROR: could not insert module /nix/store/1ibmgfr13r8b6xyn4f0wj115819f359c-linux-5.4.83/lib/modules/5.4.83/kernel/fs/efivarfs/efivarfs.ko.xz: No such device mount: /sys/firmware/efi/efivars: mount point does not exist. [ 1.908328] reboot: Power down builder for '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' failed with exit code 32 [...] Fix it by setting virtualisation.useEFIBoot = true in qemu-vm.nix, when efi is needed. And remove the now unneeded configuration in ./nixos/tests/systemd-boot.nix, since it's handled globally. Before: * release-20.03: successful build, unsuccessful run * release-20.09 (and master): unsuccessful build After: * Successful build and run. Fixes NixOS#107255

`nixos-rebuild build-vm-with-bootloader` currently fails with the default NixOS EFI configuration: $ cat >configuration.nix <<EOF { fileSystems."/".device = "/dev/sda1"; boot.loader.systemd-boot.enable = true; boot.loader.efi.canTouchEfiVariables = true; } EOF $ nixos-rebuild build-vm-with-bootloader -I nixos-config=$PWD/configuration.nix -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.09.tar.gz [...] insmod: ERROR: could not insert module /nix/store/1ibmgfr13r8b6xyn4f0wj115819f359c-linux-5.4.83/lib/modules/5.4.83/kernel/fs/efivarfs/efivarfs.ko.xz: No such device mount: /sys/firmware/efi/efivars: mount point does not exist. [ 1.908328] reboot: Power down builder for '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' failed with exit code 32 [...] Fix it by setting virtualisation.useEFIBoot = true in qemu-vm.nix, when efi is needed. And remove the now unneeded configuration in ./nixos/tests/systemd-boot.nix, since it's handled globally. Before: * release-20.03: successful build, unsuccessful run * release-20.09 (and master): unsuccessful build After: * Successful build and run. Fixes NixOS#107255 (cherry picked from commit 39fad29)

`nixos-rebuild build-vm-with-bootloader` currently fails with the default NixOS EFI configuration: $ cat >configuration.nix <<EOF { fileSystems."/".device = "/dev/sda1"; boot.loader.systemd-boot.enable = true; boot.loader.efi.canTouchEfiVariables = true; } EOF $ nixos-rebuild build-vm-with-bootloader -I nixos-config=$PWD/configuration.nix -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.09.tar.gz [...] insmod: ERROR: could not insert module /nix/store/1ibmgfr13r8b6xyn4f0wj115819f359c-linux-5.4.83/lib/modules/5.4.83/kernel/fs/efivarfs/efivarfs.ko.xz: No such device mount: /sys/firmware/efi/efivars: mount point does not exist. [ 1.908328] reboot: Power down builder for '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' failed with exit code 32 [...] Fix it by setting virtualisation.useEFIBoot = true in qemu-vm.nix, when efi is needed. And remove the now unneeded configuration in ./nixos/tests/systemd-boot.nix, since it's handled globally. Before: * release-20.03: successful build, unsuccessful run * release-20.09 (and master): unsuccessful build After: * Successful build and run. Fixes #107255 (cherry picked from commit 39fad29)

(The first version of this change, in commit 39fad29, broke `nix-build -A nixosTests.installer.simpleUefiSystemdBoot`. This is the 2nd version, which hopefully does not break anything.) `nixos-rebuild build-vm-with-bootloader` currently fails with the default NixOS EFI configuration: $ cat >configuration.nix <<EOF { fileSystems."/".device = "/dev/sda1"; boot.loader.systemd-boot.enable = true; boot.loader.efi.canTouchEfiVariables = true; } EOF $ nixos-rebuild build-vm-with-bootloader -I nixos-config=$PWD/configuration.nix -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.09.tar.gz [...] insmod: ERROR: could not insert module /nix/store/1ibmgfr13r8b6xyn4f0wj115819f359c-linux-5.4.83/lib/modules/5.4.83/kernel/fs/efivarfs/efivarfs.ko.xz: No such device mount: /sys/firmware/efi/efivars: mount point does not exist. [ 1.908328] reboot: Power down builder for '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' failed with exit code 32 [...] Fix it by setting virtualisation.useEFIBoot = true when needed. Before: * release-20.03: successful build, unsuccessful run * release-20.09 (and master): unsuccessful build After: * Successful build and run. Fixes NixOS#107255

(The first version of this change, in commit 39fad29, broke `nix-build -A nixosTests.installer.simpleUefiSystemdBoot`. This is the 2nd version, which hopefully does not break anything.) `nixos-rebuild build-vm-with-bootloader` currently fails with the default NixOS EFI configuration: $ cat >configuration.nix <<EOF { fileSystems."/".device = "/dev/sda1"; boot.loader.systemd-boot.enable = true; boot.loader.efi.canTouchEfiVariables = true; } EOF $ nixos-rebuild build-vm-with-bootloader -I nixos-config=$PWD/configuration.nix -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.09.tar.gz [...] insmod: ERROR: could not insert module /nix/store/1ibmgfr13r8b6xyn4f0wj115819f359c-linux-5.4.83/lib/modules/5.4.83/kernel/fs/efivarfs/efivarfs.ko.xz: No such device mount: /sys/firmware/efi/efivars: mount point does not exist. [ 1.908328] reboot: Power down builder for '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' failed with exit code 32 [...] Fix it by setting virtualisation.useEFIBoot = true when needed. Before: * release-20.03: successful build, unsuccessful run * release-20.09 (and master): unsuccessful build After: * Successful build and run. Fixes #107255

(The first version of this change, in commit 39fad29, broke `nix-build -A nixosTests.installer.simpleUefiSystemdBoot`. This is the 2nd version, which hopefully does not break anything.) `nixos-rebuild build-vm-with-bootloader` currently fails with the default NixOS EFI configuration: $ cat >configuration.nix <<EOF { fileSystems."/".device = "/dev/sda1"; boot.loader.systemd-boot.enable = true; boot.loader.efi.canTouchEfiVariables = true; } EOF $ nixos-rebuild build-vm-with-bootloader -I nixos-config=$PWD/configuration.nix -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.09.tar.gz [...] insmod: ERROR: could not insert module /nix/store/1ibmgfr13r8b6xyn4f0wj115819f359c-linux-5.4.83/lib/modules/5.4.83/kernel/fs/efivarfs/efivarfs.ko.xz: No such device mount: /sys/firmware/efi/efivars: mount point does not exist. [ 1.908328] reboot: Power down builder for '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' failed with exit code 32 [...] Fix it by setting virtualisation.useEFIBoot = true when needed. Before: * release-20.03: successful build, unsuccessful run * release-20.09 (and master): unsuccessful build After: * Successful build and run. Fixes #107255 (cherry picked from commit 72d906a)

bjornfor added 0.kind: bug Something is broken 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 0.kind: regression Something that worked before working no longer labels Dec 20, 2020

FRidh added this to the 21.03 milestone Dec 20, 2020

bjornfor mentioned this issue Dec 20, 2020

nixos: fix "nixos-rebuild build-vm-with-bootloader" for EFI systems #107275

Merged

10 tasks

rnhmjoj closed this as completed in #107275 Dec 21, 2020

vcunat reopened this Dec 24, 2020

bjornfor mentioned this issue Jan 5, 2021

nixos: fix "nixos-rebuild build-vm-with-bootloader" for EFI systems (v2) #108504

Merged

10 tasks

bjornfor closed this as completed in #108504 Jan 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"nixos-rebuild build-vm-with-booloader" with systemd-boot is broken again #107255

"nixos-rebuild build-vm-with-booloader" with systemd-boot is broken again #107255

bjornfor commented Dec 20, 2020 •

edited

Loading

bjornfor commented Dec 20, 2020 •

edited

Loading

bjornfor commented Dec 20, 2020

bjornfor commented Dec 20, 2020

bjornfor commented Dec 20, 2020

bjornfor commented Dec 20, 2020 •

edited

Loading

bjornfor commented Dec 20, 2020 •

edited

Loading

"nixos-rebuild build-vm-with-booloader" with systemd-boot is broken again #107255

"nixos-rebuild build-vm-with-booloader" with systemd-boot is broken again #107255

Comments

bjornfor commented Dec 20, 2020 • edited Loading

bjornfor commented Dec 20, 2020 • edited Loading

bjornfor commented Dec 20, 2020

bjornfor commented Dec 20, 2020

bjornfor commented Dec 20, 2020

bjornfor commented Dec 20, 2020 • edited Loading

bjornfor commented Dec 20, 2020 • edited Loading

bjornfor commented Dec 20, 2020 •

edited

Loading

bjornfor commented Dec 20, 2020 •

edited

Loading

bjornfor commented Dec 20, 2020 •

edited

Loading

bjornfor commented Dec 20, 2020 •

edited

Loading