| 15 Mar 2022 |
|  bobvanderlinden changed their display name from bobvanderlinden_ to bobvanderlinden. | 20:36:41 |
| bobvanderlinden | Moving the systemd helper functions to systemd-lib.nix seemed like the most important change to merge, so I create a separate PR for that: https://github.com/NixOS/nixpkgs/pull/164317 | 20:59:21 |
 @janne.hess:helsinki-systems.de | In reply to @bobvanderlinden_:matrix.org
Janne Heß: Thanks for looking into it! I completely agree. I'd like to do https://github.com/NixOS/nixpkgs/pull/164016 in stages and the individual commits are the smallest I could make the steps. A next stage is also ready, but I'd like to send that as a separate PR. That one makes the options units, services, slices, etc reusable, so they can be reused in systemd, systemd.user as well as initrd.systemd. I'd like to use the same units/services/etc structure, so that we have something to build on further.
I totally agree that stage-1.nix should be made compatible with an option to switch between the two. I worked on a local branch myself in the past where that was the case, but since this PR (https://github.com/NixOS/nixpkgs/pull/120015) seemed to be progressed further along I thought I'd resurrect that one. To move on that one further I'd like to be able to add systemd services to initrd properly, hence why I started with the refactoring of systemd. Once I can add services, I'd like to add a preDeviceCommands, postDeviceCommands, postMountCommands, etc services that will essentially emulate the current initrd boot process.
boot.initrd.systemd.enable = true; or boot.initramfs.enable = true are viable options. I still need to dive deeper into this than I currently have.
I know big PRs are quite annoying to review and merge. If I can make the changes smaller, let me know. I can also submit the individual commits as separate PRs if that's desired. For some of the changes I can even make sure that the build hashes of the tests do not change (an actual 'no functionality has changed' refactor). As long as I know the path I'm walking is one that won't get stalled (and result in neverending rebases and conflict resolution). Let me know what I can do to move things forward :+1:.
Btw, I really appreciate your message here, I was quite glad to see it. I'll install the Element mobile/desktop to keep up with notifications here.
I'll let you know if anything should be split into smaller pieces but that really depends a lot on the person reviewing it and their knowledge in the different subsystems (for example, I'd rather not merge a large PR like yours in the systemd area but I think others would be able to comprehend the implications). I usually think of PR size in a project this size of the minimal amount of changes that make sense without further changes (I hope that sentence makes sense). Another thing I learned that makes merging a lot quicker and more likely is to supply an absurd amount of tests but it's probably too early to have anything worth testing right now | 23:11:09 |
| 16 Mar 2022 |
| bobvanderlinden | Indeed. I also realized that it could take a long time before everyone agrees on all the changes that I have made in #164016. That is why I split off the most important change to https://github.com/NixOS/nixpkgs/pull/164317. I can keep working on making initramfs's systemd as nice and reusable as possible. Since it will be under a boot.initramfs.systemd.enable-option it'll be easier to merge already. Once it is settled that the reusable parts are indeed the right way to go, we can slowly merge them into the systemd and systemd.user modules. | 19:08:34 |
| bobvanderlinden | * Janne Heß: Indeed. I also realized that it could take a long time before everyone agrees on all the changes that I have made in #164016. That is why I split off the most important change to https://github.com/NixOS/nixpkgs/pull/164317. I can keep working on making initramfs's systemd as nice and reusable as possible. Since it will be under a boot.initramfs.systemd.enable-option it'll be easier to merge already. Once it is settled that the reusable parts are indeed the right way to go, we can slowly merge them into the systemd and systemd.user modules. | 19:08:43 |
| @janne.hess:helsinki-systems.de invited  Mic92. | 19:17:02 |
| Mic92 joined the room. | 19:17:12 |
| 19 Mar 2022 |
| bobvanderlinden | ElvishJerricco: my branch where I worked on your work is here: https://github.com/bobvanderlinden/nixpkgs/commits/rewrite-initrd-systemd
the most notable things I did:
- Merged nixos-unstable and made sure things are working again
- Renamed the make-initramfs-ng tool to find-dependencies. Now it doesn't copy files, but solely output paths for cpio's stdin. This should make it a bit faster.
- Added functionality to find /nix/store/XXX paths in non-ELF files. So it can recursively find dependencies in config files (like .service files).
- Make sure the nixos/tests/systemd.nix succeeded.
- Moved everything from nixos/modules/boot/stage-1.nix into nixos/modules/boot/initramfs/default.nix and nixos/modules/boot/initramfs/systemd/default.nix. This reverted stage-1.nix back to its original content, so that
boot.initrd.enable can work. This adds boot.initramfs.enable and boot.initramfs.systemd.enable.
| 09:25:08 |
 @elvishjerricco:matrix.org | Oh boy, I've got some catching up to do... | 09:26:33 |
| @elvishjerricco:matrix.org | bobvanderlinden: TBH I've basically restarted again to heavily clean up the module API so that it's almost just boot.initrd.systemd.objects and boot.initrd.systemd.<unit-type> | 09:27:25 |
| bobvanderlinden | that is mostly where I left off. The next thing I was working on was to remove systemd-units.nix and reuse the units, services, mounts from nixos/modules/boot/systemd.nix. That's why I was refactoring nixos/modules/boot/systemd.nix in a separate PR. | 09:27:49 |
| @elvishjerricco:matrix.org | Plus switching to an enable thing so that you can choose between old initrd (default) and systemd initrd | 09:27:52 |
| @elvishjerricco:matrix.org | bobvanderlinden: Yea that's what I'm doing literally right now on my branch | 09:28:17 |
| bobvanderlinden | yes, that was a good step to take. However, since we reworked the way initrd (aka initramfs) is being built, I gathered it might be safer to move that also to a separate option. | 09:29:37 |
| bobvanderlinden | reusable systemd can be found here: https://github.com/bobvanderlinden/nixpkgs/commits/pr-reusable-systemd-module | 09:29:49 |
| bobvanderlinden | I wanted to do that in stages, so that it will be easier to merge for people ☝️ | 09:30:31 |
| bobvanderlinden | Moving systemd helper functions to systemd-lib.nix: https://github.com/NixOS/nixpkgs/pull/164317
Split up systemd.nix into systemd/{user,journald,coredump,...}.nix: https://github.com/NixOS/nixpkgs/pull/164016 | 09:31:51 |
| @elvishjerricco:matrix.org | wait... is your version of the rust program copying any nix path to show up in any file? | 09:32:02 |
| @elvishjerricco:matrix.org | That's very likely to be excessive | 09:32:15 |
| bobvanderlinden | no, it only outputs to stdout the paths of the /nix/store/* files | 09:32:42 |
| bobvanderlinden | the tool doesn't write to files/directories anymore | 09:33:06 |
| @elvishjerricco:matrix.org | bobvanderlinden: Sure, but that's not what I meant | 09:34:19 |
| @elvishjerricco:matrix.org | it's just searching for any /nix/store/XXXXXXX... path in any file and adding that to the list, right? | 09:34:34 |
| bobvanderlinden | ah, yes, if it is not an ELF file, it will do that. I gathered it can be optimized a bit by detecting a text-file for instance | 09:35:20 |
| bobvanderlinden | * ah, yes, if it is not an ELF file, it will do that. I gathered it can be optimized a bit by detecting a text-file for instance, but I haven't implemented that yet. | 09:35:38 |
| @elvishjerricco:matrix.org | Hmm... I dunno whether or not that's a good idea | 09:35:39 |
| @elvishjerricco:matrix.org | It could definitely lead to excessive inclusions | 09:35:51 |
| @elvishjerricco:matrix.org | and initramfs really needs to be lean | 09:36:03 |
| @elvishjerricco:matrix.org | Not even close to as bad as copying whole nix closure of course, but I'd be curious to know if it tends to make the initramfs noticeably bigger | 09:36:55 |
| bobvanderlinden | Well, the idea was that if you refer to files in initramfs.systemd.services (directly or indirectly), it'll copy those automatically. You only need to add things to initramfs.contents if those files were missed. That shouldn't happen that often. | 09:37:32 |
