!oNSIfazDqEcwhcOjSL:matrix.org

disko

352 Members
disko - declarative disk partitioning - https://github.com/nix-community/disko90 Servers

Load older messages


SenderMessageTime
18 May 2024
@matthewcroughan:defenestrate.itmatthewcroughan

The way I see it there are only two problems remaining with disko:

  1. no compression for diskImages output
  2. no hostPlatform recognition (build on hostPlatform vm and run binfmt inside of that accelerated kvm machine)
00:09:50
@matthewcroughan:defenestrate.itmatthewcroughan2 is a bit of an architectural issue that I looked into earlier, the way disko and even vmTools are constructed, doesn't easily allow you to fix that 00:10:31
@matthewcroughan:defenestrate.itmatthewcroughanatm vmTools will fall back to tcg, and basically hang, never completing the build00:11:20
@matthewcroughan:defenestrate.itmatthewcroughaninstead, vmTools should execute a VM for the hostplatform, with binfmt enabled. But vmTools is so minimal that putting binfmt in there wouldn't re-use the nixpkgs machinery (boot.binfmt.emulatedSystems)00:12:05
@matthewcroughan:defenestrate.itmatthewcroughan * instead, vmTools should execute a VM for the hostPlatform, with binfmt enabled, and run all the scripts for arm64, But vmTools is so minimal that putting binfmt in there wouldn't re-use the nixpkgs machinery (boot.binfmt.emulatedSystems)00:12:22
@matthewcroughan:defenestrate.itmatthewcroughanThe use case is building a raspberry pi image, of course :D 00:12:34
@matthewcroughan:defenestrate.itmatthewcroughanx86 kvm -> nested arm64 binfmt, is much much much faster than arm64 -> arm64 , which is what currently happens00:15:07
@matthewcroughan:defenestrate.itmatthewcroughan *

The way I see it there are only two problems remaining with disko:

  1. no compression for diskImages output
  2. no hostPlatform recognition (build on hostPlatform vm and run binfmt for arm64 inside of that accelerated kvm machine)
00:15:32
@matthewcroughan:defenestrate.itmatthewcroughanPRs welcome, I know.. but it's really quite difficult to re-imagine the make-disk-image function and vmTools with hostPlatform conditions00:16:28
@matthewcroughan:defenestrate.itmatthewcroughan * x86 kvm -> nested arm64 binfmt, is much much much faster than arm64 binfmt -> arm64 tcg vm , which is what currently happens if you try to build an arm64 disk image on x86 00:17:12
@matthewcroughan:defenestrate.itmatthewcroughan * x86 kvm -> nested arm64 binfmt, is much much much faster than arm64 binfmt -> arm64 tcg vm , which is what currently happens if you try to build an arm64 disk image on an x86 host00:17:23
@matthewcroughan:defenestrate.itmatthewcroughan
corpo-disko-images> + findmnt /dev/disk/by-partlabel/disk-disk1-firmware /mnt/firmware
corpo-disko-images> + mount /dev/disk/by-partlabel/disk-disk1-firmware /mnt/firmware -t vfat -o defaults -o X-mount.mkdir
corpo-disko-images> + /nix/store/gc39rhvazg28s7y80c4c0a24x3hsv92p-postMountHook.sh
corpo-disko-images> + rm -rf /tmp/tmp.GkPSSeFxnb
corpo-disko-images> warning: the group 'nixbld' specified in 'build-users-group' does not exist
corpo-disko-images> cp: cannot access '/nix/store/fikh9lqcr2vfz3ncq3sihqzhjw482yps-mbrola-voices-3.3/data/gr1': Cannot allocate memory
corpo-disko-images> cp: cannot access '/nix/store/nw17gpncwy29l0r8kaxzss6nb4c1bh6s-linux-firmware-20240513-zstd/lib/firmware/qcom/sm8250': Cannot allocate memory
corpo-disko-images> [  122.745756] reboot: Power down
01:02:03
@matthewcroughan:defenestrate.itmatthewcroughan I ran into this Cannot allocate memory issue again, but it happens non-deterministically. It only happens every now and then.. very strange 01:02:29
@lassulus:lassul.uslassulusAFAIK there was a way to compress the images disko generates with a hook08:16:01
@lassulus:lassul.uslassulushttps://github.com/nix-community/disko/blob/master/lib/make-disk-image.nix#L608:16:34
19 May 2024
@matthewcroughan:defenestrate.itmatthewcroughan
In reply to @lassulus:lassul.us
https://github.com/nix-community/disko/blob/master/lib/make-disk-image.nix#L6
Oh yes, but this is more manual and not a module option, which I think it should be
11:27:02
@matthewcroughan:defenestrate.itmatthewcroughanI may figure that out and contribute it today11:27:27
@matthewcroughan:defenestrate.itmatthewcroughanI have another issue though, which is that I need the bootloader to be on a partition other than the root partition11:27:55
@matthewcroughan:defenestrate.itmatthewcroughan

This is pretty close to working

{ pkgs, ... }:
let
  configTxt = pkgs.writeText "config.txt" ''
    [pi4]
    kernel=u-boot-rpi4.bin
    enable_gic=1

    # Otherwise the resolution will be weird in most cases, compared to
    # what the pi3 firmware does by default.
    disable_overscan=1

    # Supported in newer board revisions
    arm_boost=1

    [cm4]
    # Enable host mode on the 2711 built-in XHCI USB controller.
    # This line should be removed if the legacy DWC2 controller is required
    # (e.g. for USB device mode) or if USB support is not required.
    otg_mode=1

    [all]
    # Boot in 64-bit mode.
    arm_64bit=1

    # U-Boot needs this to work, regardless of whether UART is actually used or not.
    # Look in arch/arm/mach-bcm283x/Kconfig in the U-Boot tree to see if this is still
    # a requirement in the future.
    enable_uart=1

    # Prevent the firmware from smashing the framebuffer setup done by the mainline kernel
    # when attempting to show low-voltage or overtemperature warnings.
    avoid_warnings=1
  '';
in
{
  disko.devices = {
    disk = {
      disk1 = {
        imageSize = "10G";
        type = "disk";
        device = "/dev/mmcblk0";
              postCreateHook = ''
                lsblk
                sgdisk -A 2:set:2 /dev/vdb
              '';
        content = {
          type = "gpt";
          partitions = {
            firmware = {
              size = "30M";
              type = "EF00";
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/firmware";
                postMountHook = toString (pkgs.writeScript "postMountHook.sh" ''
                  (cd ${pkgs.raspberrypifw}/share/raspberrypi/boot && cp bootcode.bin fixup*.dat start*.elf *.dtb /mnt/firmware/)
                  cp ${pkgs.ubootRaspberryPi4_64bit}/u-boot.bin /mnt/firmware/u-boot-rpi4.bin
                  cp ${configTxt} /mnt/firmware/config.txt
                '');
              };
            };
            root = {
              name = "root";
              size = "100%";
              content = {
                type = "filesystem";
                format = "bcachefs";
                mountpoint = "/";
              };
            };
          };
        };
      };
    };
  };
}


11:29:10
@matthewcroughan:defenestrate.itmatthewcroughan *

This is pretty close to working

{ pkgs, ... }:
let
  configTxt = pkgs.writeText "config.txt" ''
    [pi4]
    kernel=u-boot-rpi4.bin
    enable_gic=1

    # Otherwise the resolution will be weird in most cases, compared to
    # what the pi3 firmware does by default.
    disable_overscan=1

    # Supported in newer board revisions
    arm_boost=1

    [cm4]
    # Enable host mode on the 2711 built-in XHCI USB controller.
    # This line should be removed if the legacy DWC2 controller is required
    # (e.g. for USB device mode) or if USB support is not required.
    otg_mode=1

    [all]
    # Boot in 64-bit mode.
    arm_64bit=1

    # U-Boot needs this to work, regardless of whether UART is actually used or not.
    # Look in arch/arm/mach-bcm283x/Kconfig in the U-Boot tree to see if this is still
    # a requirement in the future.
    enable_uart=1

    # Prevent the firmware from smashing the framebuffer setup done by the mainline kernel
    # when attempting to show low-voltage or overtemperature warnings.
    avoid_warnings=1
  '';
in
{
  disko.devices = {
    disk = {
      disk1 = {
        imageSize = "10G";
        type = "disk";
        device = "/dev/mmcblk0";
              postCreateHook = ''
                lsblk
                sgdisk -A 2:set:2 /dev/vdb
              '';
        content = {
          type = "gpt";
          partitions = {
            firmware = {
              size = "30M";
              type = "EF00";
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/firmware";
                postMountHook = toString (pkgs.writeScript "postMountHook.sh" ''
                  (cd ${pkgs.raspberrypifw}/share/raspberrypi/boot && cp bootcode.bin fixup*.dat start*.elf *.dtb /mnt/firmware/)
                  cp ${pkgs.ubootRaspberryPi4_64bit}/u-boot.bin /mnt/firmware/u-boot-rpi4.bin
                  cp ${configTxt} /mnt/firmware/config.txt
                '');
              };
            };
            root = {
              name = "root";
              size = "100%";
              content = {
                type = "filesystem";
                format = "bcachefs";
                mountpoint = "/";
              };
            };
          };
        };
      };
    };
  };
}


11:29:15
@matthewcroughan:defenestrate.itmatthewcroughan Note sgdisk -A 2:set:2 /dev/vdb because of Pi/U-Boot interactions when using GPT only without a PMBR 11:29:41
@matthewcroughan:defenestrate.itmatthewcroughan U-boot requires the legacy bios bootable flag on the root partition where /boot/extlinux resides 11:30:00
@matthewcroughan:defenestrate.itmatthewcroughanbut u-boot doesn't support loading extlinux from bcachefs yet, so I need a little fat/ext4 partition containing it11:30:46
@matthewcroughan:defenestrate.itmatthewcroughansystemd-repart can't do bcachefs due to needing loopback though11:33:04
@matthewcroughan:defenestrate.itmatthewcroughanOh, in addition to compression not being a module option, there's also image resizing, which would be a really nice to have, though maybe it's not in disko's domain, I am not sure about it 11:37:37
@lassulus:lassul.uslassulusI think compression is fine as a hook. But we need more documentation for that11:37:54
@matthewcroughan:defenestrate.itmatthewcroughan Disko already "pollutes?" the NixOS Configuration with the automatic filesystems options, so maybe it should also handle disk resizing if enabled as a module option 11:38:39
@matthewcroughan:defenestrate.itmatthewcroughanthe same way systemd-repart does11:38:43
@matthewcroughan:defenestrate.itmatthewcroughan Arguably, hardware-configuration.nix and associated stuff is the real pollution, and disko/repart fix that 11:39:20
@matthewcroughan:defenestrate.itmatthewcroughanFound https://github.com/nix-community/disko/pull/384 which sorta documents the compression you can apply with extraPostVM11:41:52

Show newer messages


Back to Room ListRoom Version: 10