| 22 Jun 2023 |
 @nikstur:matrix.org | The qemu-vm module should take care of this. And even if I add the module to boot.initrd.availbleKernelModules it doesnt work. However, it works if / is NOT on a tmpfs. I can't make the connect between tmpfs and 9p failing | 10:14:40 |
| @nikstur:matrix.org | Oh and it also works with the legacy scripted initrd. So there is definetely something going on in the systemd initrd. | 10:22:42 |
 colemickens | Ooh I know this one. | 12:55:42 |
| colemickens | You need a special kernel module for PCI in qemu | 12:56:00 |
| colemickens | https://github.com/colemickens/nixcfg/blob/main/modules/other-arch-vm.nix#L68 | 12:58:20 |
| colemickens | That finally fixed the no channels available for me, I'm pretty sure | 12:58:56 |
 @lily:lily.flowers | Do we need to add that to the qemu-vm modules then? | 13:01:19 |
| colemickens | I think that might make sense, yeah | 13:01:58 |
 @gdamjan:spodeli.org | but IIRC there was an option for qemu to use the mmio virtio without pci at all? | 13:08:09 |
| @lily:lily.flowers | Seems weird it would only happen with tmpfs and only with systemd-initrd too, but idk I'd have to poke at it. Does adding that one module actually fix your issue nikstur? | 13:10:32 |
| @nikstur:matrix.org | I don't know where "pci-host-generic" comes from, but it doesn't seem to be in the normal kernel build: FATAL: Module pci-host-generic not found in directory /nix/store/w37l1l2g8ysm54ccrq4q4bj06wjhcl2y-linux-6.1.34-modules/lib/modules/6.1.34 | 13:21:46 |
| @nikstur:matrix.org | In reply to @lily:lily.flowers
Seems weird it would only happen with tmpfs and only with systemd-initrd too, but idk I'd have to poke at it. Does adding that one module actually fix your issue nikstur? I also can't make the connection there. This is where it works and doesnt:
- legacy-initrd without tmpfs / -> works
- legacy-initrd with tmpfs / -
| 13:23:00 |
| @nikstur:matrix.org | In reply to @lily:lily.flowers
Seems weird it would only happen with tmpfs and only with systemd-initrd too, but idk I'd have to poke at it. Does adding that one module actually fix your issue nikstur? * I also can't make the connection there. This is where it works and doesnt:
- legacy-initrd without tmpfs / -> works
- legacy-initrd with tmpfs / -> works
- sd-initrd wihtout tmpfs / -> works
- sd-initrd with tmpfs / -> doesn't work
| 13:23:22 |
| @nikstur:matrix.org | * I also can't make the connection there. This is where it works and doesnt:
- legacy-initrd without tmpfs / -> works
- legacy-initrd with tmpfs / -> works
- sd-initrd without tmpfs / -> works
- sd-initrd with tmpfs / -> doesn't work
| 13:23:38 |
| @lily:lily.flowers | In reply to @nikstur:matrix.org
I also can't make the connection there. This is where it works and doesnt:
- legacy-initrd without tmpfs / -> works
- legacy-initrd with tmpfs / -> works
- sd-initrd without tmpfs / -> works
- sd-initrd with tmpfs / -> doesn't work
Can you share full boot logs? Or provide a system derivation to reproduce so we can poke at it? | 13:28:07 |
| @nikstur:matrix.org | https://github.com/NixOS/nixpkgs/pull/238848 | 13:28:44 |
| @nikstur:matrix.org | Didnt get around to parameterize it for sd-initrd yet but if you just manually set boot.initrd.systemd.enable = true it fails | 13:29:09 |
| @lily:lily.flowers | If no one else does, I'll take a look and poke a little in about an hour or two when I'm done with a thing at $dayjob | 13:31:09 |
| @lily:lily.flowers | Alright so you're gonna hate this and I think it may be a kernel bug | 17:24:35 |
| @lily:lily.flowers | It works with this snippet:
boot.initrd.kernelModules = [ "9p" "9pnet_virtio" ];
boot.initrd.systemd.services.systemd-modules-load.before = [ "sysroot.mount" ];
boot.initrd.systemd.services.systemd-modules-load.serviceConfig.ExecStartPost = "${pkgs.coreutils}/bin/sleep 5";
| 17:24:52 |
| @lily:lily.flowers | Something about loading 9p after /sysroot tmpfs is mounted breaks it | 17:25:02 |
| @lily:lily.flowers | (and yes it did also need the sleep) | 17:25:10 |
| @lily:lily.flowers | nikstur | 17:25:17 |
| @lily:lily.flowers | Scripted stage-1 only works because it handles that specially and serially rather than generally and parallel like systemd-initrd | 17:25:40 |
| @nikstur:matrix.org | I figured it would be something like this... :(( | 17:37:11 |
| @lily:lily.flowers | Looks like 6.3 still has the same problem and I'm testing 5.15 now. I didn't find any immediately obvious related bug reports on lkml or bugzilla, but I also didn't look too hard and not sure which exact part of that interaction does it | 17:38:45 |
| @lily:lily.flowers | (I'll admit I really don't feel like bisecting the kernel right now, though, if it does turn out to be a kernel bug) | 17:40:16 |
| @gdamjan:spodeli.org | tmpfs /sysroot is too fast, and not the whole PCI is enumerated? | 18:05:23 |
| @lily:lily.flowers | Yeah but scripted stage-1 should be loading 9pnet_virtio on-demand too. Let me try introducing a wait between sysroot.mount and the 9pnet mounts | 18:07:26 |
| @gdamjan:spodeli.org | I don't think async PCI is on the mind of many people :D | 18:07:59 |
