!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

464 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.144 Servers

You have reached the beginning of time (for this room).


SenderMessageTime
19 Jun 2026
@joerg:thalheim.ioMic92Did the first build actually finish?05:31:00
@joerg:thalheim.ioMic92Maybe just a missing DB update 05:31:27
@k900:0upti.meK900 https://hydra.nixos.org/build/331675357 another funni 05:38:13
@joerg:thalheim.ioMic92I just checked just looks like stale db entries05:48:11
@joerg:thalheim.ioMic92There are not actually two builds running05:48:27
@joerg:thalheim.ioMic92Looks the db isn't properly updated05:49:16
@vcunat:matrix.orgVladimír ČunátThe first one did not run until the end.05:49:20
@vcunat:matrix.orgVladimír ČunátAt least the local log on the builder "is cut off".05:49:38
@joerg:thalheim.ioMic92The process does05:49:40
@joerg:thalheim.ioMic92I can't see the actual build running05:49:54
@vcunat:matrix.orgVladimír ČunátPerhaps it got killed or something.05:50:10
@joerg:thalheim.ioMic92I just did a db query and found 13 phantom builds05:52:38
@joerg:thalheim.ioMic92I don't think those builds were ever scheduled on an actual machine05:57:03
@vcunat:matrix.orgVladimír Čunát

I can see a partial log here

[root@goofy-hopcroft:~]# nix-store -l /nix/store/fdccspjycvm1lm99aimnq5fw7rrbh4ny-firefox-unwrapped-152.0.drv
06:04:04
@vcunat:matrix.orgVladimír ČunátBut there were multiple aborted attempts of that build on that machine.06:04:29
@vcunat:matrix.orgVladimír Čunát(we were restarting the queue-runner a lot, and the build takes hours)06:04:40
@leona:leona.isleonaquite a few builds are 'receiving outputs' for multiple hours now: https://hydra.nixos.org/machines on multiple machines08:12:09
@joerg:thalheim.ioMic92I can check again, but yesterday it was genuine compression cpu load it had to do and it was competing with build resources08:28:36
@joerg:thalheim.ioMic92we are using the highest zstd compression ratio08:28:51
@leona:leona.isleona

3h for etc.drv is much..

08:31:57
@leona:leona.isleona * >3h for etc.drv is much.. 08:32:05
@joerg:thalheim.ioMic92Has to wait. I am currently working on something else08:35:19
@joerg:thalheim.ioMic92Deploying to hydra-staging now to sanity check my scheduler chhanges08:52:15

Show newer messages


Back to Room ListRoom Version: 6