Sender | Message | Time |
---|---|---|
31 Oct 2023 | ||
l0b0 | * I'm starting to think some unreliable test runs — "waiting for the VM to finish booting" never being paired up with a "(finished: waiting for the VM to finish booting, in N.M seconds)" — could be caused by KVM not being set up:
I don't know which level this happens at, but I'm running the | 02:46:36 |
l0b0 | * I'm starting to think some unreliable test runs — "waiting for the VM to finish booting" never being paired up with a "(finished: waiting for the VM to finish booting, in N.M seconds)" — could be caused by KVM not being set up:
I don't know which level this happens at, but I'm running the | 02:53:07 |
K900 | Gitlab managed runners aren't guaranteed to have KVM AFAIK | 07:55:16 |
l0b0 | In reply to @k900:0upti.meOh, OK. I couldn't find a definitive answer for this anywhere, so that makes sense. | 08:59:21 |
K900 | They have it sometimes IME | 08:59:30 |
K900 | But they use multiple cloud providers for those | 08:59:46 |
l0b0 | Still, I have no idea why my tests are failing randomly, and even less how to fix it now. | 08:59:46 |
K900 | Well if there's no KVM it will fall back to software emulation | 09:00:28 |
K900 | Which is not fast | 09:00:29 |
K900 | So it'll probably just time out | 09:00:39 |
l0b0 | Oh, by the way, they don't always fail when KVM is missing. | 09:00:51 |
l0b0 | So it can't be as simple as that, either. | 09:01:05 |
K900 | Well they can succeed if the CPU is fast enough | 09:01:35 |
K900 | To actually run the JIT at a reasonable speed | 09:01:42 |
l0b0 | I haven't found a single common theme that separates 8-minute successes from 3-hour failures, except for the obvious one, that nixosTests' start_all never seems to finish during failed runs. | 09:02:20 |
l0b0 | And that's despite both nodes running happily, spitting out journalctl rotation messages every few minutes. | 09:02:50 |
l0b0 | So I'm starting to be more inclined towards start_all being buggy, and not actually detecting the system startup properly. | 09:03:28 |
raitobezarius | To debug this properly, you'd need to attach all sorts of logs from the firmware, the kernel and QEMU itself | 09:53:42 |
raitobezarius | Otherwise this is all speculation | 09:53:47 |
l0b0 | Oof | 10:26:11 |
1 Nov 2023 | ||
l0b0 | In reply to @raitobezarius:matrix.orgAny pointers to do any of this? I've no idea whether this is even possible on GitLab. | 05:59:08 |
raitobezarius | Well if it's a VM, you can enable debugging of OVMF if you are using UEFI, you can modify the scripts to make QEMU log information about itself and you can boot the kernel with the right log level, send this as artifacts and bring them | 11:00:17 |
Ramses 🇵🇸 changed their display name from Ramses to Ramses 🇵🇸. | 19:58:28 | |
2 Nov 2023 | ||
l0b0 | In reply to @raitobezarius:matrix.orgI guess none of that is possible in GitLab, since I can't control how anything outside the Nix container runs. | 04:31:43 |
l0b0 | How do I run multiple NixOS tests in one go? If I use nixosLib.runTest with a separate import per test, the imports conflict. | 05:32:50 |
l0b0 | * How do I run multiple NixOS tests in one go? If I use nixosLib.runTest with a separate import per test, the imports conflict because they are setting the same attributes. But there's no runTests in /nixos/lib . | 05:33:28 |
K900 | Tests are just derivations | 05:34:44 |
l0b0 | Yep | 05:34:56 |
K900 | So you can symlinkJoin them or whatever | 05:35:00 |
l0b0 | Isn't there a standard pattern for this? | 05:35:32 |