!PSmBFWNKoXmlQBzUQf:helsinki-systems.de

Stage 1 systemd

83 Members
systemd in NixOs's stage 1, replacing the current bash tooling https://github.com/NixOS/nixpkgs/projects/5127 Servers

Load older messages


SenderMessageTime
16 Apr 2023
@hexa:lossy.network@hexa:lossy.network * If system-wide link configs are applied in stage1 that would make my use case for flushing non-existant20:28:29
@flokli:matrix.orgflokliAs far as other netdev settings (like controlling link speeds explicitly, or *{CheckSum,Segmentation}Offload, systemd also pretty much added all these as options20:32:53
@flokli:matrix.orgflokli * As far as other netdev settings (like controlling link speeds explicitly, or *{CheckSum,Segmentation}Offload, systemd also pretty much added all these as options. I'm not sure how much of that is exposed to be accessible from the module system 20:33:06
@flokli:matrix.orgflokliand when it comes to "having a different IP address in initrd to avoid host key stuff", I'd probably go with a different port, and keep the IP. But in any case, that might be a usecase that could be mentioned in the description20:35:18
17 Apr 2023
@flokli:matrix.orgflokli ElvishJerricco: sorry for being so pedantic on the networking bits in the PR, but i feel like we need to get the semantics sorted out before too many people end up switching to it, and possibly get angry if we switch semantics and break possibly remote machines. 21:08:14
@elvishjerricco:matrix.org@elvishjerricco:matrix.orgYep, I totally understand21:08:29
@elvishjerricco:matrix.org@elvishjerricco:matrix.orgI do really want to get it done before 23.05, but that shouldn't be too hard even with max pedantry :)21:08:50
@flokli:matrix.orgflokliI'll try to check my notifications more frequently in the coming days :-)21:09:30
@elvishjerricco:matrix.org@elvishjerricco:matrix.org

Rather than continuing to discuss this tangent on the PR,

As for the link units, the PR that added this comment also linked to an issue, but that issue was closed by this way before that PR was even made. I'm wondering why that systemd change isn't enough to resolve the problem. Surely you're not expected to do your interface renaming in stage 1. In fact, the comments in the issue suggest that the udev rules to rename the interface in stage 2 should happen before networkd is even notified the interface is ready.

21:18:09
@elvishjerricco:matrix.org@elvishjerricco:matrix.org And fwiw, if I remove the line that adds the link units to systemd stage 1 and enable systemd stage 1 in the nixosTests.networking.rename test, the test doesn't fail even after 20 tries. 21:19:59
@elvishjerricco:matrix.org@elvishjerricco:matrix.org * And fwiw, if I remove the line that adds the link units to systemd stage 1 and enable systemd stage 1 in the nixosTests.networking.networkd.rename test, the test doesn't fail even after 20 tries. 21:20:16
@lily:lily.flowers@lily:lily.flowers
In reply to @elvishjerricco:matrix.org
And fwiw, if I remove the line that adds the link units to systemd stage 1 and enable systemd stage 1 in the nixosTests.networking.networkd.rename test, the test doesn't fail even after 20 tries.
Yeah I tried it as well and that test seemed perfectly happy renaming in stage2. But I admittedly didn't try it with initrd networking enabled
21:47:26
18 Apr 2023
@nikstur:matrix.org@nikstur:matrix.org joined the room.09:26:29
@nikstur:matrix.org@nikstur:matrix.org
In reply to @elvishjerricco:matrix.org
like, if we're going to require the stage 2 fs to be set up, then we should do it in stage 2, not stage 1. And if we're doing it in stage 1, just put the config files in stage 1
This is, however, just the design of systemd-repart. Not a decision the module makes: https://www.freedesktop.org/software/systemd/man/systemd-repart.html#--root= systemd-repart is really designed to work with disk images (containing all required data to setup the system, including the root fs). Running systemd-repart in the initrd is then just a matter of running it earlier so that other services that start in stage 2 can already work on the enlarged partitions. Also the module gives the user the choice to run repart either in the initrd or in stage 2. All in all it to mimic the standard behaviour of repart as closely as possible in NixOS.
09:38:23
@nikstur:matrix.org@nikstur:matrix.org
In reply to @elvishjerricco:matrix.org
like, if we're going to require the stage 2 fs to be set up, then we should do it in stage 2, not stage 1. And if we're doing it in stage 1, just put the config files in stage 1
* This is, however, just the design of systemd-repart. Not a decision the module makes: https://www.freedesktop.org/software/systemd/man/systemd-repart.html#--root= systemd-repart is really designed to work with disk images (containing all required data to setup the system, including the root fs). Running systemd-repart in the initrd is then just a matter of running it earlier so that other services that start in stage 2 can already work on the enlarged partitions. Also the module gives the user the choice to run repart either in the initrd or in stage 2. All in all it tries to mimic the standard behaviour of repart as closely as possible in NixOS.
10:02:04
@elvishjerricco:matrix.org@elvishjerricco:matrix.org

This is, however, just the design of systemd-repart

No? The docs say nothing of the sort. --root is just something that a lot of systemd utils support; it's not saying that's the only way to work with not-the-current-root-fs

The use case for systemd-repart in stage 1 is 100% for formatting and resizing the root fs. This is a major reason why /sysusr exists at all; systemd-repart finds the config files in what will become /usr and uses that to format what will become /. /sysusr exists so that /sysroot doesn't have to be mounted first, precisely because things like systemd-repart are meant to alter that stuff

16:41:25
@elvishjerricco:matrix.org@elvishjerricco:matrix.org There is no use case where systemd-repart alters only non-root disks and cannot be run in stage 2 and ordered before whatever services depend on it. 16:44:20
@elvishjerricco:matrix.org@elvishjerricco:matrix.org And there's absolutely no point in running it in stage 1 if running it requires initrd-fs.target to be reached 16:46:27
@elvishjerricco:matrix.org@elvishjerricco:matrix.org (the nixos module just requires sysroot.mount to be reached but if this is the design we're going with it really should wait for /sysroot/nix/store, and the simplest way is initrd-fs.target) 16:47:44
@nikstur:matrix.org@nikstur:matrix.org

The docs say this is the default: "By default when invoked on the regular system this defaults to the host's root file system /. If invoked from the initrd this defaults to /sysroot/, so that the tool operates on the configuration and machine ID stored in the root file system later transitioned into itself.". I tried to stay as close as possible to the default.

systemd-repart actually does not only look into sysusr. It also looks for configs in /sysroot/etc. E.g. see the conditions for the service:

ConditionDirectoryNotEmpty=|/sysroot/usr/lib/repart.d
ConditionDirectoryNotEmpty=|/sysroot/usr/local/lib/repart.d
ConditionDirectoryNotEmpty=|/sysroot/etc/repart.d
ConditionDirectoryNotEmpty=|/sysusr/usr/lib/repart.d
ConditionDirectoryNotEmpty=|/sysusr/usr/local/lib/repart.d

If I understand it correctly, your point is that the nix store might not be on sysroot but instead mounted later (alongside the other partitions marked with x-initrd.mount in the fstab). Thus, it would only be available after initrs-fs.target is reached. Then instead of sysroot.mount, we should use initrd-fs.target to order the systemd-repart service. That makes sense to me.

I also agree that /sysusr/ is the preferred/ideal way to store the config files in the systemd world. However, I don't see a way to imitate that in NixOS.

On the top of my head I can think of at least one solid use case where systemd-repart has to run in the initrd. Imagine you have disk images that only contain the Nix Store on the root fs without even space in the root fs to accomodate the rest of the / subtree created during activation. Repart can then make the space in the root fs after sysroot.mount (or even initrd-fs.target) but before activation.

I don't really love baking configuration (i.e. the repart configs) into the initrd if it can be avoided. And again, I think the current module is closer to how systemd-repart operates by default than the alternative of baking the configs into the initrd. But maybe we can give users the option in the module.

21:22:01
@nikstur:matrix.org@nikstur:matrix.orgSorry for the long ass reply. This was a good exercise to think it all through again :) 21:22:38
@nikstur:matrix.org@nikstur:matrix.org *

The docs say this is the default: "By default when invoked on the regular system this defaults to the host's root file system /. If invoked from the initrd this defaults to /sysroot/, so that the tool operates on the configuration and machine ID stored in the root file system later transitioned into itself.". I tried to stay as close as possible to the default.

systemd-repart actually does not only look into sysusr. It also looks for configs in /sysroot/etc. E.g. see the conditions for the service:

ConditionDirectoryNotEmpty=|/sysroot/usr/lib/repart.d
ConditionDirectoryNotEmpty=|/sysroot/usr/local/lib/repart.d
ConditionDirectoryNotEmpty=|/sysroot/etc/repart.d
ConditionDirectoryNotEmpty=|/sysusr/usr/lib/repart.d
ConditionDirectoryNotEmpty=|/sysusr/usr/local/lib/repart.d

If I understand it correctly, your point is that the nix store might not be on sysroot but instead mounted later (alongside the other partitions marked with x-initrd.mount in the fstab). Thus, it would only be available after initrs-fs.target is reached. Then instead of sysroot.mount, we should use initrd-fs.target to order the systemd-repart service. That makes sense to me.

I also agree that /sysusr/ is the preferred/ideal way to store the config files in the systemd world. However, I don't see a way to imitate that in NixOS.

On the top of my head I can think of at least one solid use case where systemd-repart has to run in the initrd. Imagine you have disk images that only contain the Nix Store on the root fs without even space in the root fs to accomodate the rest of the / subtree created during activation. Repart can then make the space in the root fs after sysroot.mount (or even initrd-fs.target) but before activation.

I don't really love baking configuration (i.e. the repart configs) into the initrd if it can be avoided. And again, I think the current module is closer to how systemd-repart operates by default than the alternative of baking the configs into the initrd. But maybe we can give users the option in the module.

21:25:01
@elvishjerricco:matrix.org@elvishjerricco:matrix.org

If invoked from the initrd this defaults to /sysroot/, so that the tool operates on the configuration and machine ID stored in the root file system later transitioned into itself

I think you're leaning heavily on this line as the only indication of how the docs want it to work in initrd. But I still think it's a misinterpretation. By defaulting to /sysroot, that includes the required disk formatting, and therefore comes before sysroot.mount and not after.

21:39:12
@elvishjerricco:matrix.org@elvishjerricco:matrix.org I'm made more confident about this given a comment I received from poettering suggesting that root fs formatting is the job of repart 21:40:46
@elvishjerricco:matrix.org@elvishjerricco:matrix.org

On the top of my head I can think of at least one solid use case where systemd-repart has to run in the initrd. Imagine you have disk images that only contain the Nix Store on the root fs without even space in the root fs to accomodate the rest of the / subtree created during activation. Repart can then make the space in the root fs after sysroot.mount (or even initrd-fs.target) but before activation.

Partitions can often be resized online. e.g. Look at how systemd-growfs works. The partition table is not part of the FS, so that can be changed without consequence. Then the FS can be adjusted to consume the whole partition online ala growfs

21:45:07
@elvishjerricco:matrix.org@elvishjerricco:matrix.org (in fact that's how fileSystems.<name>.autoResize works with systemd initrd enabled) 21:46:40
@elvishjerricco:matrix.org@elvishjerricco:matrix.org I maintain that there is no excuse for mounting the root FS before running systemd-repart in initrd. 21:47:46
@elvishjerricco:matrix.org@elvishjerricco:matrix.orgah wait I misunderstood the example21:49:29
@elvishjerricco:matrix.org@elvishjerricco:matrix.orghmm21:49:51
@elvishjerricco:matrix.org@elvishjerricco:matrix.org well that use case would still be taken care of by running systemd-repart before mounting /sysroot 21:50:39

Show newer messages


Back to Room ListRoom Version: 6