!DBFhtjpqmJNENpLDOv:nixos.org

NixOS systemd

579 Members
NixOS ❤️ systemd159 Servers

Load older messages


SenderMessageTime
24 Mar 2025
@p14:matrix.orgp14

Thanks for the hint; could this explain the oddity I'm seeing?

Essentially, I have a ro store, that I mount at /nix/rostore, and then a writable partition I mount at /. I want to make /nix/store be an overlayfs with /nix/store (on the writable partition) with the rostore as the lower layer. I have something that works, but then the /nix/store directory appears empty if you try to list the files in it. Directly opening the paths is fine.

I can't figure out why I can't get a sane overlayfs mount..

18:53:14
@elvishjerricco:matrix.org@elvishjerricco:matrix.org p14: the quickest guidance I can think of is that I did this for the ISO here: https://github.com/NixOS/nixpkgs/pull/291750 18:54:14
@p14:matrix.orgp14Essentially I'm trying to do something like https://github.com/NixOS/nixpkgs/blob/8e3b25f1708783d02963efe47222adb0a8e6f4f7/nixos/modules/installer/cd-dvd/iso-image.nix#L81318:54:17
@p14:matrix.orgp14Oh cool, will definitely unpack taht18:54:36
@elvishjerricco:matrix.org@elvishjerricco:matrix.org p14: oh, actually, I forgot it was already basically correct in the ISO 18:55:05
@elvishjerricco:matrix.org@elvishjerricco:matrix.orgwhat's wrong with doing it the way the ISO does it already?18:55:15
@p14:matrix.orgp14I guess one difference there is that / is tmpfs18:55:19
@p14:matrix.orgp14I don't want / as tmpfs18:55:22
@elvishjerricco:matrix.org@elvishjerricco:matrix.orgthat doesn't really matter18:55:29
@p14:matrix.orgp14Well, OK, but then if I try to do what I perceive to be the obvious thing, I end up with this weird thing which doesn't quite work properly18:55:50
@elvishjerricco:matrix.org@elvishjerricco:matrix.orgcan you share the code you tried and what you observed going wrong?18:56:24
@p14:matrix.orgp14 I also had difficulty getting the debug shell to work with my VM since I can't switch to VT9, though I did just discover rd.systemd.default_debug_tty, so once I can figure out what the friendly tty is called, I can probably get a debug shell 18:56:27
@p14:matrix.orgp14

Something like:

fileSystems = {
  "/" = {
    device = "/dev/disk/by-partlabel/nixos-rw";
    fsType = "btrfs";
    options = [ "x-systemd.growfs" ];
    autoResize = true;
    neededForBoot = true;
  };
  "/nix/rostore" = {
    device = "/dev/disk/by-partlabel/nixos";
    fsType = "squashfs";
    neededForBoot = true;
  };
  "/nix/store" = {
    fsType = "overlay";
    device = "overlay";
    overlay = {
      lowerdir = ["/nix/rostore/nix/store"];
      upperdir = "/nix/store";
      workdir = "/nix/ovfs-work";
    };
  };
};
18:59:16
@p14:matrix.orgp14 Awkwardly my rostore has the nix store at /nix/store within it 18:59:41
@elvishjerricco:matrix.org@elvishjerricco:matrix.org p14: You don't want the upper dir to be /nix/store 19:01:47
@elvishjerricco:matrix.org@elvishjerricco:matrix.orgThat should be a separate directory19:01:53
@p14:matrix.orgp14Ah, that'd probably be it. I was operating under the belief that this would be the upperdir from before this one was mounted, not so, eh?19:05:35
@p14:matrix.orgp14 Looks like did the trick ElvishJerricco , thank you for rescuing me from my own insanity <3 19:10:50
@elvishjerricco:matrix.org@elvishjerricco:matrix.org it would be if you were just running mount manually. But the overlay module in nixos creates systemd dependencies on the file systems for the upper/work/lower dirs, so I think you were creating a cyclic dependency of /nix/store on itself which probably blew something up 19:12:18
@elvishjerricco:matrix.org@elvishjerricco:matrix.org i.e. I think it probably resulted in an fstab line like overlay /nix/store overlay ...,x-systemd.requires-mounts-for=/nix/store 19:13:12
@elvishjerricco:matrix.org@elvishjerricco:matrix.org I guess it would work if you didn't use the nixos overlay module and instead just manually wrote the options 19:13:54
@p14:matrix.orgp14Well it didn't work great when I did that either. At least having the writable stuff go to a different place is working well enough19:14:48
@p14:matrix.orgp14Now for my next challenge, make my x86 image also legacy bootable 🤦. I'd love to use the systemd-repart image builder for this but maybe that is too much to ask.19:15:52
@p14:matrix.orgp14I'm wondering if I can get away with building an image with the grub MBR and use repart to put that image in the right place. But then I'm not really clear on whether I can mark this partition as bootable, I'm guessing not since this is not the systemd way.19:16:50
@p14:matrix.orgp14(the systemd way being UEFI boot, because, obviously)19:17:00
@elvishjerricco:matrix.org@elvishjerricco:matrix.org p14: so, repart will not mark a partition as LegacyBIOSBootable, in fdisk speak. That's a GPT flag though, and I don't think it's strictly necessary for BIOS booting; I think the MBR just has to be done right. Nonetheless, I do have a repart patch for allowing setting that GPT flag with its numeric identifier: https://gist.github.com/ElvishJerricco/6059640087e1ee09744bb9df8f18bb1a 19:22:34
@elvishjerricco:matrix.org@elvishjerricco:matrix.orgbut, getting an MBR configured properly for the image using repart is probably nontrivial and I have no idea how to do that19:23:08
@elvishjerricco:matrix.org@elvishjerricco:matrix.org well, I mean I know repart won't help 19:23:22
@elvishjerricco:matrix.org@elvishjerricco:matrix.orgbut I mean letting repart make the GPT image, and then separately writting the MBR on LBA 019:23:57
@elvishjerricco:matrix.org@elvishjerricco:matrix.orgno idea how to do that for grub19:24:01

Show newer messages


Back to Room ListRoom Version: 6