 | NixOS AWS | 65 Members |
| 16 Servers |
| NixOS AWS | 65 Members |
| 16 Servers |
| Sender | Message | Time |
|---|---|---|
| 6 Sep 2024 | ||
 Arian | hmm what kind of bootstrapping assumptions do you mean? | 07:54:25 |
| Arian | the upload-ami script takes any NixOS disk image. How you build it is up to you. The current github action downloads the image from Hydra; which builds the image similarly to: https://github.com/NixOS/amis/pull/166/files | 07:55:14 |
| Arian | You should be able to build the image on any machine | 07:55:28 |
| Arian | that has nix installed | 07:55:34 |
| Arian | The current iteration of the image builder works by spawning KVM inside a nix build for some config. Next iteration will use systemd-repart which will mean a completely VM-less build | 07:56:09 |
| Arian | so one bootstrap problem is: Your CI needs to support KVM. Which is problematic as AWS doesn’t support KVM | 07:56:20 |
| Arian | but I’m working on it | 07:56:26 |
| Arian | Then plan is to move to https://nixos.org/manual/nixos/stable/#sec-image-repart which doesn’t require KVM | 07:56:50 |
| Arian | ACtually KVM is not needed. But the build will be a bit slower than without KVM | 07:57:21 |
| Arian | the build works fine on AWS. it will just use emulated qemu instead of KVM qemu for assembling the final image | 07:57:33 |
| Arian | makes the build a few seconds slower | 07:57:40 |
| Arian | ah you’re calling make-disk-image too. | 08:00:21 |
| Arian | so yeh. doing the same thing! | 08:00:26 |
 commiterate | I guess some explicit requirements on the machines and software needed to get the image build pipeline working in the first place. e.g. I have this: https://github.com/commiterate/nix-images?tab=readme-ov-file#bootstrap | 17:28:39 |
| commiterate | In reply to @arianvp:matrix.orgOh nice, time to try this out then to remove nested virtualization (via KVM) from the bootstrap requirements then | 17:29:55 |
| commiterate | The reason I'm particularly sensitive to bootstrap is related to the GitLab runner fleet I mentioned in the AWSTOE issue. With GitHub, there's GitHub-hosted runners which provide some "fixed"/"stable" bootstrap machines (i.e. x86-64 and AArch64 Linux runners). With a private GitLab instance, you have no runners to start. For ease of deployability, I also can't assume too much of the initial system used to build up any IaC templates to deploy bootstrap infra to AWS or whatever. | 17:32:33 |
| commiterate | * The reason I'm particularly sensitive to bootstrap is related to the GitLab runner fleet I mentioned in the AWSTOE issue. With GitHub, there's GitHub-hosted runners which provide some "fixed"/"stable" bootstrap machines (i.e. x86-64 and AArch64 Linux runners). With a private GitLab instance, you have no runners to start. For ease of deployability, I also can't expect too much from the initial system used to build up any IaC templates to deploy bootstrap infra to AWS or whatever (i.e. shouldn't expect cross-compile or emulated native compile capabilities since these may be Darwin or Windows machines). | 17:33:16 |
| commiterate | In reply to @arianvp:matrix.orgI believe this is only true on virts (instances using the Nitro Hypervisor). Metals have direct hardware access so as long as the AMI used there support KVM, it's usable. Not that it's a hard requirement for make-disk-image like you said since QEMU can just swap the backend. | 17:43:54 |
| commiterate | In reply to @arianvp:matrix.org* I believe this is only true on virts (instances using the Nitro Hypervisor). Metals have direct hardware access so as long as the AMI used there supports KVM, it's usable. Not that it's a hard requirement for make-disk-image like you said since QEMU can just swap the backend. | 17:44:48 |
| 7 Sep 2024 | ||
| commiterate | Hmm playing around with
Only step 2 requires booting up a VM. Essentially, it takes the disk image from step 1 and then runs NixOS install steps. Step 1 doesn't require a VM just like
| 00:56:55 |
| commiterate | * Hmm playing around with
Only step 2 requires booting up a VM. Essentially, it takes the disk image from step 1 and then runs NixOS install steps. Step 1 doesn't require a VM just like
| 00:58:56 |
| commiterate | * Hmm playing around with
Only step 2 requires booting up a VM. Essentially, it takes the disk image from step 1 and then runs NixOS install steps. Step 1 doesn't require a VM just like
| 01:00:23 |
| commiterate | Maybe there's something I'm misunderstanding about the UKI that should be included in appliances. https://nixos.org/manual/nixos/stable/#sec-image-repart-appliance There's this repo which seems to be producing a working image that the author is using: https://github.com/tnytown/nixos-lima-vm Trying to do the same thing and launching the | 02:16:07 |
| commiterate | * Maybe there's something I'm misunderstanding about the UKI that should be included in appliances. Maybe it includes systemd and some other stuff that removes the need for the https://nixos.org/manual/nixos/stable/#sec-image-repart-appliance There's this repo which seems to be producing a working image that the author is using: https://github.com/tnytown/nixos-lima-vm Trying to do the same thing and launching the | 02:16:54 |
| commiterate | * It seems there's something I'm misunderstanding about the UKI that should be included in appliances. Maybe it includes systemd and some other stuff that removes the need for the https://nixos.org/manual/nixos/stable/#sec-image-repart-appliance There's this repo which seems to be producing a working image that the author is using: https://github.com/tnytown/nixos-lima-vm Trying to do the same thing and launching the | 02:22:06 |
| commiterate | Hmm even the current image I'm generating with make-nix-image hangs, so either something's wrong with both my images or my QEMU setup is broken. | 03:05:09 |
| commiterate | * Hmm even the current image I'm generating with make-nix-image hangs, so either something's wrong with both my images or my QEMU setup is broken. NixOS test VMs seem to work fine though (since I was able to run the one used in the amazon-cloudwatch-agent NixOS tests fine). | 03:05:37 |
| commiterate | * Hmm playing around with
Only step 2 requires booting up a VM and is optional if you don't need a full NixOS install. Essentially, it takes the disk image from step 1 and then runs NixOS install steps. Step 1 doesn't require a VM just like
| 16:58:36 |
| commiterate | * Hmm playing around with
Only step 2 requires booting up a VM and is optional if you don't need a full NixOS install. Essentially, it takes the disk image from step 1 and then runs NixOS install steps. Step 1 doesn't require a VM just like
| 16:58:50 |
| commiterate | Nevermind, it's because I disabled hardware virtualization awhile back in an attempt to fix some stability problems. Turning it back on and forcing QEMU to use UEFI lets it proceed. | 18:26:17 |