| 18 Jun 2026 |
 Mic92 | * Looks I got it fixed now with the new parallel upload code: https://hydra.nixos.org/machines all machines are building and rarely upload | 19:59:22 |
| Mic92 | * Looks I got it fixed now with the new parallel upload code: https://hydra.nixos.org/machines all machines are building and rarely uploading | 19:59:35 |
|  Tobi joined the room. | 20:25:50 |
| 19 Jun 2026 |
 Vladimír Čunát | Same derivation is being built on two machines at once? At a glance, this happens even in a situation where the previous queue-runner wouldn't do it:
https://hydra.nixos.org/build/331659169#tabs-buildsteps | 03:28:10 |
| Mic92 | Did the first build actually finish? | 05:31:00 |
| Mic92 | Maybe just a missing DB update | 05:31:27 |
 K900 | https://hydra.nixos.org/build/331675357 another funni | 05:38:13 |
| Mic92 | I just checked just looks like stale db entries | 05:48:11 |
| Mic92 | There are not actually two builds running | 05:48:27 |
| Mic92 | Looks the db isn't properly updated | 05:49:16 |
| Vladimír Čunát | The first one did not run until the end. | 05:49:20 |
| Vladimír Čunát | At least the local log on the builder "is cut off". | 05:49:38 |
| Mic92 | The process does | 05:49:40 |
| Mic92 | I can't see the actual build running | 05:49:54 |
| Vladimír Čunát | Perhaps it got killed or something. | 05:50:10 |
| Mic92 | I just did a db query and found 13 phantom builds | 05:52:38 |
| Mic92 | I don't think those builds were ever scheduled on an actual machine | 05:57:03 |
| Vladimír Čunát | I can see a partial log here
[root@goofy-hopcroft:~]# nix-store -l /nix/store/fdccspjycvm1lm99aimnq5fw7rrbh4ny-firefox-unwrapped-152.0.drv
| 06:04:04 |
| Vladimír Čunát | But there were multiple aborted attempts of that build on that machine. | 06:04:29 |
| Vladimír Čunát | (we were restarting the queue-runner a lot, and the build takes hours) | 06:04:40 |
 leona | quite a few builds are 'receiving outputs' for multiple hours now: https://hydra.nixos.org/machines on multiple machines | 08:12:09 |
| Mic92 | I can check again, but yesterday it was genuine compression cpu load it had to do and it was competing with build resources | 08:28:36 |
| Mic92 | we are using the highest zstd compression ratio | 08:28:51 |
| leona |
3h for etc.drv is much..
| 08:31:57 |
| leona | * >3h for etc.drv is much.. | 08:32:05 |
| Mic92 | Has to wait. I am currently working on something else | 08:35:19 |
| Mic92 | Deploying to hydra-staging now to sanity check my scheduler chhanges | 08:52:15 |
| Mic92 | * Deploying to hydra-staging now to sanity check my scheduler changes | 08:58:32 |
 hexa | full disk on mimas o_O | 10:13:14 |
| hexa | https://grafana.nixos.org/d/rYdddlPWk/node-exporter-full?orgId=1&from=now-2d&to=now&timezone=browser&var-datasource=default&var-job=node&var-node=mimas.nixos.org:9100&var-diskdevices=%5Ba-z%5D%2B%7Cnvme%5B0-9%5D%2Bn%5B0-9%5D%2B%7Cmmcblk%5B0-9%5D%2B&refresh=1m&viewPanel=panel-152 | 10:13:56 |
