| 28 Sep 2025 |
 John Ericson | Attach debugger to nix while builder is sleeping, then trace to the part where it kills the builder | 12:18:56 |
| John Ericson | * 3. Attach debugger to nix while builder is sleeping, then trace to the part where it kills the builder | 12:19:06 |
 Mic92 | Is nix a separate process in this case? | 16:08:22 |
| Mic92 | There are also tricks where a function spawns gdb to itself. Also i would use gdbstub here | 16:09:16 |
 vcunat | At a quick glance, maybe you'd consider to use rr to record an execution and then inspect that recording (which happens in a gdb interface again). | 17:16:52 |
| vcunat | * At a quick glance, maybe you'd consider using rr to record an execution and then inspect that recording (which happens in a gdb interface again). | 17:17:07 |
| John Ericson | In reply to @joerg:thalheim.io
Is nix a separate process in this case? With the ssh://localhost yes | 23:11:04 |
| John Ericson | Could use ?remote-program=... to run GDB hah | 23:11:49 |
| John Ericson | In reply to @vcunat:matrix.org
At a quick glance, maybe you'd consider using rr to record an execution and then inspect that recording (which happens in a gdb interface again). Yeah that might work well with the remote program thing | 23:12:15 |
| 29 Sep 2025 |
| Mic92 | John Ericson: we need to make the call soon. I would prefer to get rid of nix 2.29 in nixpkgs soon to not have to maintain it for another release. | 06:27:25 |
| 30 Sep 2025 |
| vcunat | Apparently it's flaky. Sometimes I get {"status":"unknown"}, sometimes a proper reply. | 09:13:26 |
 Janne | run3(['hydra-queue-runner', '--status'], \undef, \$stdout, \$stderr);
my $status;
if ($? != 0) {
$status = { status => "unknown" };
} else {
$status = decode_json($stdout);
}
| 10:58:12 |
| Janne | that happens when we cannot execute the status command | 10:58:18 |
| Janne | should be fixed with the new queue runner when we expose the metrics directly rather than going through catalyst. that would require a specific proxyPass in nginx but should simplify everything and make it more stable | 10:59:35 |
| vcunat | 🤔 so now I tried to run the command manually on that particular machine, repeatedly. Sometimes after the correct-looking JSON it added
error: queue runner did not respond; status information may be wrong
and exited with status 1.
| 11:06:41 |
| vcunat | And apparently the time to process the command fluctuates significantly. | 11:07:09 |
| vcunat | (sometimes immediate, sometimes a couple seconds) | 11:07:44 |
| vcunat | OK, I suppose it's not worth delving into details for this on the old queue runner. | 11:08:24 |
 Sandro | Do you have an idea since which version it is broken? | 11:54:31 |
|  @saiko:knifepoint.net left the room. | 19:27:48 |
| 3 Oct 2025 |
|  @bootstrapper:matrix.org removed their profile picture. | 10:08:15 |
| @bootstrapper:matrix.org removed their display name Ido Samuelson. | 10:08:33 |
| @bootstrapper:matrix.org left the room. | 10:08:55 |
|  moleksiak joined the room. | 23:21:01 |
| 5 Oct 2025 |
| John Ericson | @joerg:thalheim.io: OK let's just comment out that test for now | 12:25:51 |
| John Ericson | I don't have time for more things | 12:26:00 |
| John Ericson | Still would like to do 2.30 then 2.31 however | 12:26:14 |
| John Ericson | Can get the test passing again with the new queue runner :) | 12:26:38 |
| Mic92 | Yep | 12:32:05 |
| Mic92 | Hopefully also easier to debug | 12:32:15 |
