!jtzdZrQSXtEpeGtzAn:nixos.org

Testing with Nix

64 Members
22 Servers

Load older messages


SenderMessageTime
31 Oct 2023
@vengmark2:matrix.orgl0b0 *

I'm starting to think some unreliable test runs — "waiting for the VM to finish booting" never being paired up with a "(finished: waiting for the VM to finish booting, in N.M seconds)" — could be caused by KVM not being set up:

Virtualisation disk image created.
Could not access KVM kernel module: No such file or directory
qemu-kvm: failed to initialize kvm: No such file or directory
qemu-kvm: falling back to tcg

I don't know which level this happens at, but I'm running the nixos/nix:2.18.1 container and setting system-features = benchmark big-parallel kvm nixos-test. What else do I need to do to enable KVM?

02:46:36
@vengmark2:matrix.orgl0b0 *

I'm starting to think some unreliable test runs — "waiting for the VM to finish booting" never being paired up with a "(finished: waiting for the VM to finish booting, in N.M seconds)" — could be caused by KVM not being set up:

Virtualisation disk image created.
Could not access KVM kernel module: No such file or directory
qemu-kvm: failed to initialize kvm: No such file or directory
qemu-kvm: falling back to tcg

I don't know which level this happens at, but I'm running the nixos/nix:2.18.1 container and setting system-features = benchmark big-parallel kvm nixos-test. What else do I need to do to enable KVM? /dev/kvm does not exist within the Docker image.

02:53:07
@k900:0upti.meK900Gitlab managed runners aren't guaranteed to have KVM AFAIK07:55:16
@vengmark2:matrix.orgl0b0
In reply to @k900:0upti.me
Gitlab managed runners aren't guaranteed to have KVM AFAIK
Oh, OK. I couldn't find a definitive answer for this anywhere, so that makes sense.
08:59:21
@k900:0upti.meK900 They have it sometimes IME 08:59:30
@k900:0upti.meK900But they use multiple cloud providers for those08:59:46
@vengmark2:matrix.orgl0b0Still, I have no idea why my tests are failing randomly, and even less how to fix it now.08:59:46
@k900:0upti.meK900Well if there's no KVM it will fall back to software emulation09:00:28
@k900:0upti.meK900Which is not fast09:00:29
@k900:0upti.meK900So it'll probably just time out09:00:39
@vengmark2:matrix.orgl0b0 Oh, by the way, they don't always fail when KVM is missing. 09:00:51
@vengmark2:matrix.orgl0b0So it can't be as simple as that, either.09:01:05
@k900:0upti.meK900Well they can succeed if the CPU is fast enough09:01:35
@k900:0upti.meK900To actually run the JIT at a reasonable speed09:01:42
@vengmark2:matrix.orgl0b0 I haven't found a single common theme that separates 8-minute successes from 3-hour failures, except for the obvious one, that nixosTests' start_all never seems to finish during failed runs. 09:02:20
@vengmark2:matrix.orgl0b0And that's despite both nodes running happily, spitting out journalctl rotation messages every few minutes.09:02:50
@vengmark2:matrix.orgl0b0 So I'm starting to be more inclined towards start_all being buggy, and not actually detecting the system startup properly. 09:03:28
@raitobezarius:matrix.orgraitobezariusTo debug this properly, you'd need to attach all sorts of logs from the firmware, the kernel and QEMU itself09:53:42
@raitobezarius:matrix.orgraitobezariusOtherwise this is all speculation09:53:47
@vengmark2:matrix.orgl0b0Oof10:26:11
1 Nov 2023
@vengmark2:matrix.orgl0b0
In reply to @raitobezarius:matrix.org
To debug this properly, you'd need to attach all sorts of logs from the firmware, the kernel and QEMU itself
Any pointers to do any of this? I've no idea whether this is even possible on GitLab.
05:59:08
@raitobezarius:matrix.orgraitobezariusWell if it's a VM, you can enable debugging of OVMF if you are using UEFI, you can modify the scripts to make QEMU log information about itself and you can boot the kernel with the right log level, send this as artifacts and bring them11:00:17
@rvdp:infosec.exchangeRamses 🇵🇸 changed their display name from Ramses to Ramses 🇵🇸.19:58:28
2 Nov 2023
@vengmark2:matrix.orgl0b0
In reply to @raitobezarius:matrix.org
Well if it's a VM, you can enable debugging of OVMF if you are using UEFI, you can modify the scripts to make QEMU log information about itself and you can boot the kernel with the right log level, send this as artifacts and bring them
I guess none of that is possible in GitLab, since I can't control how anything outside the Nix container runs.
04:31:43
@vengmark2:matrix.orgl0b0 How do I run multiple NixOS tests in one go? If I use nixosLib.runTest with a separate import per test, the imports conflict. 05:32:50
@vengmark2:matrix.orgl0b0 * How do I run multiple NixOS tests in one go? If I use nixosLib.runTest with a separate import per test, the imports conflict because they are setting the same attributes. But there's no runTests in /nixos/lib. 05:33:28
@k900:0upti.meK900Tests are just derivations05:34:44
@vengmark2:matrix.orgl0b0Yep05:34:56
@k900:0upti.meK900 So you can symlinkJoin them or whatever 05:35:00
@vengmark2:matrix.orgl0b0Isn't there a standard pattern for this?05:35:32

Show newer messages


Back to Room ListRoom Version: 9