| 26 Oct 2023 |
 Robert Hensing (roberth) | OOM on the host would give an EOF, not hang | 15:10:58 |
 raitobezarius | no kernel panic will just hang | 15:11:00 |
| raitobezarius | UEFI panic would hang too | 15:11:11 |
 l0b0 | In reply to @raitobezarius:matrix.org
UEFI panic would hang too It's pretty random (~80% of runs), so it would be really weird if somehow I managed to trigger something that low-level. | 19:29:28 |
| l0b0 | In reply to @raitobezarius:matrix.org
no kernel panic will just hang Do you mean "No, kernel panic will just hang" or "No kernel panic will ever just hang"? | 19:29:57 |
| l0b0 | Maybe the test runner is running out of RAM, and ends up swapping? If so, it would be useful to start the NixOS VMs with less RAM. (Off to the search engines!) | 19:31:54 |
| l0b0 | Looks like this is easily configurable, nice! I'll give that a try. | 19:34:08 |
|  @lehmanator:gnulinux.club joined the room. | 20:33:53 |
| 27 Oct 2023 |
| l0b0 | The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. | 01:00:35 |
| l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. Let's see if GitLab agrees… | 01:01:01 |
|  @federicodschonborn:matrix.org changed their profile picture. | 01:24:48 |
| l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢 | 01:55:58 |
| l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. | 01:56:43 |
| l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. Also doesn't seem to be working. Maybe the problem is effectively reversed, and it's the nodes which are running low on memory, even though there is no information to that effect? Testing with 2048 MiB per node… | 02:09:56 |
| l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked (locally). So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. Also doesn't seem to be working. Maybe the problem is effectively reversed, and it's the nodes which are running low on memory, even though there is no information to that effect? Testing with 2048 MiB per node… | 02:14:31 |
| l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked (locally). So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. Also doesn't seem to be working. Maybe the problem is effectively reversed, and it's the nodes which are running low on memory, even though there is no information to that effect? Testing with 2048 MiB per node… Also nope 😢. WTF is going on? | 02:23:22 |
| l0b0 | It looks like nc never returns (it has no timeout by default), meaning the 900 second timeout never triggers. Forcing a timeout with a horrible hack might work. If so, I'd like to implement https://github.com/NixOS/nixpkgs/issues/157195 to work around this more permanently. | 06:01:30 |
| l0b0 | Redacted or Malformed Event | 08:33:30 |
| raitobezarius | I think you are misunderstanding that we have no guarantee Python is running inside the VM | 08:34:00 |
| l0b0 | Oh right, only on the orchestrator. My bad. | 08:34:38 |
|  K900 changed their profile picture. | 08:34:56 |
| K900 | How am I not in this room | 08:35:05 |
| raitobezarius | to protect you | 08:35:13 |
| l0b0 | Feck. That does mean I need to keep digging into arcane shell tools to fix my tests. | 08:36:07 |
|  pbsds joined the room. | 12:40:08 |
|  Ramses 🇵🇸 joined the room. | 22:53:36 |
| 29 Oct 2023 |
| raitobezarius | Robert Hensing (roberth): do you have leftover concerns for the timeout PR or would you allow me to send it? | 15:56:05 |
| 31 Oct 2023 |
| l0b0 | I'm starting to think some unreliable test runs —
waiting for the VM to finish booting
never finishing — could be caused by KVM not being set up:
Virtualisation disk image created.
Could not access KVM kernel module: No such file or directory
qemu-kvm: failed to initialize kvm: No such file or directory
qemu-kvm: falling back to tcg
| 02:43:42 |
| l0b0 | * I'm starting to think some unreliable test runs — "waiting for the VM to finish booting" never being paired up with a "(finished: waiting for the VM to finish booting, in N.M seconds)" — could be caused by KVM not being set up:
Virtualisation disk image created.
Could not access KVM kernel module: No such file or directory
qemu-kvm: failed to initialize kvm: No such file or directory
qemu-kvm: falling back to tcg
| 02:44:48 |
| l0b0 | * I'm starting to think some unreliable test runs — "waiting for the VM to finish booting" never being paired up with a "(finished: waiting for the VM to finish booting, in N.M seconds)" — could be caused by KVM not being set up:
Virtualisation disk image created.
Could not access KVM kernel module: No such file or directory
qemu-kvm: failed to initialize kvm: No such file or directory
qemu-kvm: falling back to tcg
I don't know which level this happens at, but I'm running the nixos/nix:2.18.1 container and setting system-features = benchmark big-parallel kvm nixos-test.
| 02:46:14 |
