| 28 Jun 2026 |
ghpzin | There seems to be some funny thing with systemd 261 and systemd-boot-builder.py: https://github.com/NixOS/nixpkgs/blob/08bbeb4f13347cb830f1ad61628165c5b93c77a8/nixos/modules/system/boot/loader/systemd-boot/systemd-boot-builder.py#L491-L505 with 261 introducing fallbacks in: https://github.com/systemd/systemd/commit/918c592272fb4e93b99997362a8c9ecc31e77036 there are now 2 entries and fallback is always above (at least on my machine), so regex pick it:
Available Boot Loaders on ESP:
ESP: /boot (/dev/disk/by-partuuid/b6f5a9f2-8aa9-430e-8ca6-dfdc61aecdc0)
File: ├─/boot//EFI/systemd/systemd-boot-fallbackx64.efi (systemd-boot 260.1)
├─/boot//EFI/systemd/systemd-bootx64.efi (systemd-boot 261)
└─/boot//EFI/BOOT/BOOTX64.EFI (systemd-boot 261)
and every nixos-rebuild switch attempt fails with:
updating systemd-boot from 260.1 to 261
Skipping "/boot/EFI/systemd/systemd-bootx64.efi", same boot loader version in place already.
Skipping "/boot/EFI/BOOT/BOOTX64.EFI", same boot loader version in place already.
Traceback (most recent call last):
File "/nix/store/0v7rngj7lhw6fnbpvimaxcwrmd2sly4h-systemd-boot/bin/systemd-boot", line 682, in <module>
main()
~~~~^^
File "/nix/store/0v7rngj7lhw6fnbpvimaxcwrmd2sly4h-systemd-boot/bin/systemd-boot", line 660, in main
install_bootloader(args)
~~~~~~~~~~~~~~~~~~^^^^^^
File "/nix/store/0v7rngj7lhw6fnbpvimaxcwrmd2sly4h-systemd-boot/bin/systemd-boot", line 526, in install_bootloader
run(
~~~^
[f"{SYSTEMD}/bin/bootctl", f"--esp-path={EFI_SYS_MOUNT_POINT}"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+ bootctl_flags
^^^^^^^^^^^^^^^
+ ["update"]
^^^^^^^^^^^^
)
^
File "/nix/store/0v7rngj7lhw6fnbpvimaxcwrmd2sly4h-systemd-boot/bin/systemd-boot", line 243, in run
return subprocess.run(cmd, check=True, text=True, stdout=stdout, stderr=sys.stderr)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/jxyrvv4gbpnp3ap5iy7wxwl1sg4x2x88-python3-3.14.6/lib/python3.14/subprocess.py", line 578, in run
raise CalledProcessError(retcode, process.args,
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/nix/store/k0k82yla4xvxzx3hbc9gaqbfm67hmqfk-systemd-261/bin/bootctl', '--esp-path=/boot', 'update']' returned non-zero exit status 1.
Failed to install bootloader
| 15:19:04 |
ghpzin | There seems to be some funny thing with systemd 261 and systemd-boot-builder.py: https://github.com/NixOS/nixpkgs/blob/08bbeb4f13347cb830f1ad61628165c5b93c77a8/nixos/modules/system/boot/loader/systemd-boot/systemd-boot-builder.py#L491-L505 with 261 introducing fallbacks in: https://github.com/systemd/systemd/commit/918c592272fb4e93b99997362a8c9ecc31e77036 there are now 2 entries and fallback is always above (at least on my machine), so regex pick it:
Available Boot Loaders on ESP:
ESP: /boot (/dev/disk/by-partuuid/b6f5a9f2-8aa9-430e-8ca6-dfdc61aecdc0)
File: ├─/boot//EFI/systemd/systemd-boot-fallbackx64.efi (systemd-boot 260.1)
├─/boot//EFI/systemd/systemd-bootx64.efi (systemd-boot 261)
└─/boot//EFI/BOOT/BOOTX64.EFI (systemd-boot 261)
and every nixos-rebuild attempt without --install-bootloader fails with:
updating systemd-boot from 260.1 to 261
Skipping "/boot/EFI/systemd/systemd-bootx64.efi", same boot loader version in place already.
Skipping "/boot/EFI/BOOT/BOOTX64.EFI", same boot loader version in place already.
Traceback (most recent call last):
File "/nix/store/0v7rngj7lhw6fnbpvimaxcwrmd2sly4h-systemd-boot/bin/systemd-boot", line 682, in <module>
main()
~~~~^^
File "/nix/store/0v7rngj7lhw6fnbpvimaxcwrmd2sly4h-systemd-boot/bin/systemd-boot", line 660, in main
install_bootloader(args)
~~~~~~~~~~~~~~~~~~^^^^^^
File "/nix/store/0v7rngj7lhw6fnbpvimaxcwrmd2sly4h-systemd-boot/bin/systemd-boot", line 526, in install_bootloader
run(
~~~^
[f"{SYSTEMD}/bin/bootctl", f"--esp-path={EFI_SYS_MOUNT_POINT}"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+ bootctl_flags
^^^^^^^^^^^^^^^
+ ["update"]
^^^^^^^^^^^^
)
^
File "/nix/store/0v7rngj7lhw6fnbpvimaxcwrmd2sly4h-systemd-boot/bin/systemd-boot", line 243, in run
return subprocess.run(cmd, check=True, text=True, stdout=stdout, stderr=sys.stderr)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/jxyrvv4gbpnp3ap5iy7wxwl1sg4x2x88-python3-3.14.6/lib/python3.14/subprocess.py", line 578, in run
raise CalledProcessError(retcode, process.args,
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/nix/store/k0k82yla4xvxzx3hbc9gaqbfm67hmqfk-systemd-261/bin/bootctl', '--esp-path=/boot', 'update']' returned non-zero exit status 1.
Failed to install bootloader
| 15:25:38 |
K900 | That's not what's happening here | 15:25:57 |
K900 | It's bootctl update that fails | 15:26:03 |
K900 | Which is very sus | 15:26:05 |
ghpzin | Iirc that is a very old thing. Skips exit with status code 1, unless you add --graceful | 15:29:25 |
K900 | hexa do you know why our python3Packages.plexapi is ancient | 15:36:17 |
hexa | will look in a bit, fixing opentelemetry mess first | 15:36:43 |
K900 | Oh wait | 15:40:17 |
K900 | It's specifically ancient plexapi pinned in HA | 15:40:22 |
hexa | then I know why | 15:43:11 |
hexa | lmao | 15:43:13 |
K900 | I'm guessing the why is "HA lol" | 15:43:21 |
ElvishJerricco | There was a previous update where bootctl update was changed to iterate over all files in $ESP/EFI/BOOT/ to see if they're systemd-boot binaries that needed updating. But it had, earlier in the code, already directly decided to update the main BOOTX64.EFI loader in there. So it got to that file, saw it was a systemd-boot binary, and tried to update it. But when bootctl update sees that a binary it tried to update is already up to date, it exits non-zero. So it was just unconditionally exiting non-zero because, even when it did update the file, it would try to update it a second time in the same invocation and obviously that's already up to date.
Anyway, this is an old bug that was fixed a long time ago, and is not necessarily related at all. I'm mentioning it because I would be very unsurprised if there was another goof a lot like that, where bootctl-install.c is just bugging out because of spaghetti logic
| 15:43:27 |
K900 | And it doesn't build because it wants pkg_resources | 15:43:37 |
hexa | setuptools_80 | 15:43:44 |
hexa | but not suitable for propagation | 15:44:01 |
K900 | It wants it at setup.py time | 15:44:10 |
hexa | good | 15:44:14 |
K900 | So HOPEFULLY I can just chuck it into nativeBuildInputs | 15:44:17 |
ElvishJerricco | (unfortunately I don't think I can look into it today) | 15:44:30 |
ghpzin | Yes, that is what I remember it from: https://github.com/NixOS/nixpkgs/issues/331170 https://github.com/systemd/systemd/issues/33392
In this case because of regex version check after first update all other invocation actually don't update anything (both non-fallback files are up to date). So I think from systemd side it is correct ? | 15:49:30 |
Randy Eckenrode | I reverted it locally and can confirm glib built again. | 15:50:50 |
ghpzin | Yes, that is what I remember it from: https://github.com/NixOS/nixpkgs/issues/331170 https://github.com/systemd/systemd/issues/33392
In this case because of regex version check after first update all other invocation actually don't update anything (both non-fallback files are up to date). So I think from systemd side it is correct (update exits with 1 if it updated nothing) ? | 15:51:14 |
ghpzin | Yes, that is what I remember it from: https://github.com/NixOS/nixpkgs/issues/331170 https://github.com/systemd/systemd/issues/33392
In this case because of regex version check after first update all other invocation actually don't update anything (both non-fallback files are up to date). So I think from systemd side it is correct (update exits with 1 if it updated nothing). | 15:52:43 |
ElvishJerricco | ah, I see, I misunderstood your previous message a bit. If the issue is just that the regex picks the wrong file, that's probably our issue. Really wish there was just a bootctl check command to check if an update was needed | 15:57:06 |
ElvishJerricco | actually | 15:57:07 |
ElvishJerricco | can we just patch that in and submit it upstream? | 15:57:21 |
ElvishJerricco | that seems way better, assuming they'll take it upstream | 15:57:29 |
ElvishJerricco | which it seems like something reasonable for them to have | 15:57:39 |