Sender | Message | Time |
---|---|---|
27 Oct 2023 | ||
* The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. | 01:56:43 | |
* The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked. So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. Also doesn't seem to be working. Maybe the problem is effectively reversed, and it's the nodes which are running low on memory, even though there is no information to that effect? Testing with 2048 MiB per node… | 02:09:56 | |
* The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked (locally). So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. Also doesn't seem to be working. Maybe the problem is effectively reversed, and it's the nodes which are running low on memory, even though there is no information to that effect? Testing with 2048 MiB per node… | 02:14:31 | |
* The default is 1 GiB. Reducing to 128 MiB broke it (OOM during boot), but 256 MiB worked (locally). So 512 MiB should be safe. Let's see if GitLab agrees… Nope 😢; testing with 256 MiB. Also doesn't seem to be working. Maybe the problem is effectively reversed, and it's the nodes which are running low on memory, even though there is no information to that effect? Testing with 2048 MiB per node… Also nope 😢. WTF is going on? | 02:23:22 | |
It looks like nc never returns (it has no timeout by default), meaning the 900 second timeout never triggers. Forcing a timeout with a horrible hack might work. If so, I'd like to implement https://github.com/NixOS/nixpkgs/issues/157195 to work around this more permanently. | 06:01:30 | |
Redacted or Malformed Event | 08:33:30 | |
I think you are misunderstanding that we have no guarantee Python is running inside the VM | 08:34:00 | |
Oh right, only on the orchestrator. My bad. | 08:34:38 | |
08:34:56 | ||
How am I not in this room | 08:35:05 | |
to protect you | 08:35:13 | |
Feck. That does mean I need to keep digging into arcane shell tools to fix my tests. | 08:36:07 | |
12:40:08 | ||
22:53:36 | ||
29 Oct 2023 | ||
Robert Hensing (roberth): do you have leftover concerns for the timeout PR or would you allow me to send it? | 15:56:05 | |
31 Oct 2023 | ||
I'm starting to think some unreliable test runs —
never finishing — could be caused by KVM not being set up:
| 02:43:42 | |
* I'm starting to think some unreliable test runs — "waiting for the VM to finish booting" never being paired up with a "(finished: waiting for the VM to finish booting, in N.M seconds)" — could be caused by KVM not being set up:
| 02:44:48 | |
* I'm starting to think some unreliable test runs — "waiting for the VM to finish booting" never being paired up with a "(finished: waiting for the VM to finish booting, in N.M seconds)" — could be caused by KVM not being set up:
I don't know which level this happens at, but I'm running the | 02:46:14 | |
* I'm starting to think some unreliable test runs — "waiting for the VM to finish booting" never being paired up with a "(finished: waiting for the VM to finish booting, in N.M seconds)" — could be caused by KVM not being set up:
I don't know which level this happens at, but I'm running the | 02:46:36 | |
* I'm starting to think some unreliable test runs — "waiting for the VM to finish booting" never being paired up with a "(finished: waiting for the VM to finish booting, in N.M seconds)" — could be caused by KVM not being set up:
I don't know which level this happens at, but I'm running the | 02:53:07 | |
Gitlab managed runners aren't guaranteed to have KVM AFAIK | 07:55:16 | |
In reply to @k900:0upti.meOh, OK. I couldn't find a definitive answer for this anywhere, so that makes sense. | 08:59:21 | |
They have it sometimes IME | 08:59:30 | |
But they use multiple cloud providers for those | 08:59:46 | |
Still, I have no idea why my tests are failing randomly, and even less how to fix it now. | 08:59:46 | |
Well if there's no KVM it will fall back to software emulation | 09:00:28 | |
Which is not fast | 09:00:29 | |
So it'll probably just time out | 09:00:39 | |
Oh, by the way, they don't always fail when KVM is missing. | 09:00:51 | |
So it can't be as simple as that, either. | 09:01:05 |