24 Oct 2023 |
Robert Hensing (roberth) | that should give you <test>.config.rawTestDerivation to feed into testBuildFailure | 10:11:11 |
raitobezarius | OK this is also what I understood | 10:11:31 |
raitobezarius | Perfect | 10:11:33 |
25 Oct 2023 |
| Federico Damián Schonborn changed their profile picture. | 00:13:16 |
l0b0 | What's the "Guest root shell did not produce any data yet..." message when running NixOS tests? I think I understand the basic principle (something is waiting for output), but Google wasn't helpful in figuring out what. | 07:02:15 |
raitobezarius | In reply to @vengmark2:matrix.org What's the "Guest root shell did not produce any data yet..." message when running NixOS tests? I think I understand the basic principle (something is waiting for output), but Google wasn't helpful in figuring out what. it means that the test driver is waiting for bytes from the guest via the serial socket | 11:45:11 |
raitobezarius | the spam was recently fixed | 11:45:27 |
raitobezarius | it happens whenever the serial socket returns empty bytes | 11:45:33 |
raitobezarius | hm | 11:45:52 |
raitobezarius | or maybe not | 11:45:54 |
raitobezarius | it also happens when there is no output for a while | 11:45:58 |
raitobezarius | e.g. long computations | 11:46:04 |
l0b0 | Thanks, that leaves the question of why this has suddenly started happening. The obvious one would be OOM killed, but I don't see any mention in the logs of that triggering. | 19:09:18 |
26 Oct 2023 |
Robert Hensing (roberth) | don't kernel panics just hang, or do we configure the kernel not to do that and halt instead? | 15:10:46 |
Robert Hensing (roberth) | OOM on the host would give an EOF, not hang | 15:10:58 |
raitobezarius | no kernel panic will just hang | 15:11:00 |
raitobezarius | UEFI panic would hang too | 15:11:11 |
l0b0 | In reply to @raitobezarius:matrix.org UEFI panic would hang too It's pretty random (~80% of runs), so it would be really weird if somehow I managed to trigger something that low-level. | 19:29:28 |
l0b0 | In reply to @raitobezarius:matrix.org no kernel panic will just hang Do you mean "No, kernel panic will just hang" or "No kernel panic will ever just hang"? | 19:29:57 |
l0b0 | Maybe the test runner is running out of RAM, and ends up swapping? If so, it would be useful to start the NixOS VMs with less RAM. (Off to the search engines!) | 19:31:54 |
l0b0 | Looks like this is easily configurable, nice! I'll give that a try. | 19:34:08 |
| Sam Lehman joined the room. | 20:33:53 |
27 Oct 2023 |
l0b0 | The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. | 01:00:35 |
l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. Let's see if GitLab agrees… | 01:01:01 |
| Federico Damián Schonborn changed their profile picture. | 01:24:48 |
l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢 | 01:55:58 |
l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. | 01:56:43 |
l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. Also doesn't seem to be working. Maybe the problem is effectively reversed, and it's the nodes which are running low on memory, even though there is no information to that effect? Testing with 2048 MiB per node… | 02:09:56 |
l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked (locally). So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. Also doesn't seem to be working. Maybe the problem is effectively reversed, and it's the nodes which are running low on memory, even though there is no information to that effect? Testing with 2048 MiB per node… | 02:14:31 |
l0b0 | * The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked (locally). So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. Also doesn't seem to be working. Maybe the problem is effectively reversed, and it's the nodes which are running low on memory, even though there is no information to that effect? Testing with 2048 MiB per node… Also nope 😢. WTF is going on? | 02:23:22 |