| 16 May 2024 |
nhnn | In reply to@lassulus:lassul.us That creates an image file with a disko defined partition layout and some extra files you pass to it It just spins up NixOS VM and creates partitions inside? | 08:18:32 |
lassulus | Yes | 08:19:34 |
nhnn | Got it, thanks. | 08:20:26 |
| 17 May 2024 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | hehe | 19:50:50 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | nix build .#nixosConfigurations.pi.config.system.build.image
warning: Git tree '/home/matthew/git/nix.how/clients/machines' is dirty
trace: warning: system.stateVersion is not set, defaulting to 24.05. Read why this matters on https://nixos.org/manual/nixos/stable/options.html#opt-system.stateVersion.
error: builder for '/nix/store/7mlf74xlxwhy4jq4rvbnjazpx5kd8bpl-image.drv' failed with exit code 1;
last 7 log lines:
> structuredAttrs is enabled
> Running phase: patchPhase
> 'repart.d' -> '/build/amended-repart.d'
> Running phase: updateAutotoolsGnuConfigScriptsPhase
> Running phase: buildPhase
> Building image with systemd-repart...
> unshare: unshare failed: Invalid argument
For full logs, run 'nix log /nix/store/7mlf74xlxwhy4jq4rvbnjazpx5kd8bpl-image.drv'.
| 19:50:52 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | bullish for disko, repart doesn't work in the binfmt emulated context | 19:51:01 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | * bullish for disko, systemd-repart doesn't work in the binfmt emulated context | 19:51:13 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | https://github.com/NixOS/nixpkgs/issues/311978 | 19:51:53 |
| 18 May 2024 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | The way I see it there are only two problems remaining with disko:
- no compression for diskImages output
- no hostPlatform recognition (build on hostPlatform vm and run binfmt inside of that accelerated kvm machine)
| 00:09:50 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | 2 is a bit of an architectural issue that I looked into earlier, the way disko and even vmTools are constructed, doesn't easily allow you to fix that | 00:10:31 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | atm vmTools will fall back to tcg, and basically hang, never completing the build | 00:11:20 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | instead, vmTools should execute a VM for the hostplatform, with binfmt enabled. But vmTools is so minimal that putting binfmt in there wouldn't re-use the nixpkgs machinery (boot.binfmt.emulatedSystems) | 00:12:05 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | * instead, vmTools should execute a VM for the hostPlatform, with binfmt enabled, and run all the scripts for arm64, But vmTools is so minimal that putting binfmt in there wouldn't re-use the nixpkgs machinery (boot.binfmt.emulatedSystems) | 00:12:22 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | The use case is building a raspberry pi image, of course :D | 00:12:34 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | x86 kvm -> nested arm64 binfmt, is much much much faster than arm64 -> arm64 , which is what currently happens | 00:15:07 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | * The way I see it there are only two problems remaining with disko:
- no compression for diskImages output
- no hostPlatform recognition (build on hostPlatform vm and run binfmt for arm64 inside of that accelerated kvm machine)
| 00:15:32 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | PRs welcome, I know.. but it's really quite difficult to re-imagine the make-disk-image function and vmTools with hostPlatform conditions | 00:16:28 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | * x86 kvm -> nested arm64 binfmt, is much much much faster than arm64 binfmt -> arm64 tcg vm , which is what currently happens if you try to build an arm64 disk image on x86 | 00:17:12 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | * x86 kvm -> nested arm64 binfmt, is much much much faster than arm64 binfmt -> arm64 tcg vm , which is what currently happens if you try to build an arm64 disk image on an x86 host | 00:17:23 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | corpo-disko-images> + findmnt /dev/disk/by-partlabel/disk-disk1-firmware /mnt/firmware
corpo-disko-images> + mount /dev/disk/by-partlabel/disk-disk1-firmware /mnt/firmware -t vfat -o defaults -o X-mount.mkdir
corpo-disko-images> + /nix/store/gc39rhvazg28s7y80c4c0a24x3hsv92p-postMountHook.sh
corpo-disko-images> + rm -rf /tmp/tmp.GkPSSeFxnb
corpo-disko-images> warning: the group 'nixbld' specified in 'build-users-group' does not exist
corpo-disko-images> cp: cannot access '/nix/store/fikh9lqcr2vfz3ncq3sihqzhjw482yps-mbrola-voices-3.3/data/gr1': Cannot allocate memory
corpo-disko-images> cp: cannot access '/nix/store/nw17gpncwy29l0r8kaxzss6nb4c1bh6s-linux-firmware-20240513-zstd/lib/firmware/qcom/sm8250': Cannot allocate memory
corpo-disko-images> [ 122.745756] reboot: Power down
| 01:02:03 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | I ran into this Cannot allocate memory issue again, but it happens non-deterministically. It only happens every now and then.. very strange | 01:02:29 |
lassulus | AFAIK there was a way to compress the images disko generates with a hook | 08:16:01 |
lassulus | https://github.com/nix-community/disko/blob/master/lib/make-disk-image.nix#L6 | 08:16:34 |
| 19 May 2024 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | In reply to @lassulus:lassul.us https://github.com/nix-community/disko/blob/master/lib/make-disk-image.nix#L6 Oh yes, but this is more manual and not a module option, which I think it should be | 11:27:02 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | I may figure that out and contribute it today | 11:27:27 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | I have another issue though, which is that I need the bootloader to be on a partition other than the root partition | 11:27:55 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | This is pretty close to working
{ pkgs, ... }:
let
configTxt = pkgs.writeText "config.txt" ''
[pi4]
kernel=u-boot-rpi4.bin
enable_gic=1
# Otherwise the resolution will be weird in most cases, compared to
# what the pi3 firmware does by default.
disable_overscan=1
# Supported in newer board revisions
arm_boost=1
[cm4]
# Enable host mode on the 2711 built-in XHCI USB controller.
# This line should be removed if the legacy DWC2 controller is required
# (e.g. for USB device mode) or if USB support is not required.
otg_mode=1
[all]
# Boot in 64-bit mode.
arm_64bit=1
# U-Boot needs this to work, regardless of whether UART is actually used or not.
# Look in arch/arm/mach-bcm283x/Kconfig in the U-Boot tree to see if this is still
# a requirement in the future.
enable_uart=1
# Prevent the firmware from smashing the framebuffer setup done by the mainline kernel
# when attempting to show low-voltage or overtemperature warnings.
avoid_warnings=1
'';
in
{
disko.devices = {
disk = {
disk1 = {
imageSize = "10G";
type = "disk";
device = "/dev/mmcblk0";
postCreateHook = ''
lsblk
sgdisk -A 2:set:2 /dev/vdb
'';
content = {
type = "gpt";
partitions = {
firmware = {
size = "30M";
type = "EF00";
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/firmware";
postMountHook = toString (pkgs.writeScript "postMountHook.sh" ''
(cd ${pkgs.raspberrypifw}/share/raspberrypi/boot && cp bootcode.bin fixup*.dat start*.elf *.dtb /mnt/firmware/)
cp ${pkgs.ubootRaspberryPi4_64bit}/u-boot.bin /mnt/firmware/u-boot-rpi4.bin
cp ${configTxt} /mnt/firmware/config.txt
'');
};
};
root = {
name = "root";
size = "100%";
content = {
type = "filesystem";
format = "bcachefs";
mountpoint = "/";
};
};
};
};
};
};
};
}
| 11:29:10 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | * This is pretty close to working
{ pkgs, ... }:
let
configTxt = pkgs.writeText "config.txt" ''
[pi4]
kernel=u-boot-rpi4.bin
enable_gic=1
# Otherwise the resolution will be weird in most cases, compared to
# what the pi3 firmware does by default.
disable_overscan=1
# Supported in newer board revisions
arm_boost=1
[cm4]
# Enable host mode on the 2711 built-in XHCI USB controller.
# This line should be removed if the legacy DWC2 controller is required
# (e.g. for USB device mode) or if USB support is not required.
otg_mode=1
[all]
# Boot in 64-bit mode.
arm_64bit=1
# U-Boot needs this to work, regardless of whether UART is actually used or not.
# Look in arch/arm/mach-bcm283x/Kconfig in the U-Boot tree to see if this is still
# a requirement in the future.
enable_uart=1
# Prevent the firmware from smashing the framebuffer setup done by the mainline kernel
# when attempting to show low-voltage or overtemperature warnings.
avoid_warnings=1
'';
in
{
disko.devices = {
disk = {
disk1 = {
imageSize = "10G";
type = "disk";
device = "/dev/mmcblk0";
postCreateHook = ''
lsblk
sgdisk -A 2:set:2 /dev/vdb
'';
content = {
type = "gpt";
partitions = {
firmware = {
size = "30M";
type = "EF00";
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/firmware";
postMountHook = toString (pkgs.writeScript "postMountHook.sh" ''
(cd ${pkgs.raspberrypifw}/share/raspberrypi/boot && cp bootcode.bin fixup*.dat start*.elf *.dtb /mnt/firmware/)
cp ${pkgs.ubootRaspberryPi4_64bit}/u-boot.bin /mnt/firmware/u-boot-rpi4.bin
cp ${configTxt} /mnt/firmware/config.txt
'');
};
};
root = {
name = "root";
size = "100%";
content = {
type = "filesystem";
format = "bcachefs";
mountpoint = "/";
};
};
};
};
};
};
};
}
| 11:29:15 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | Note sgdisk -A 2:set:2 /dev/vdb because of Pi/U-Boot interactions when using GPT only without a PMBR | 11:29:41 |
matthewcroughan @ 39c3 (DECT 94667 or 97340 or 67192) | U-boot requires the legacy bios bootable flag on the root partition where /boot/extlinux resides | 11:30:00 |