!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

470 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.146 Servers

Load older messages


SenderMessageTime
19 Jun 2026
@vcunat:matrix.orgvcunat

I can see a partial log here

[root@goofy-hopcroft:~]# nix-store -l /nix/store/fdccspjycvm1lm99aimnq5fw7rrbh4ny-firefox-unwrapped-152.0.drv
06:04:04
@vcunat:matrix.orgvcunatBut there were multiple aborted attempts of that build on that machine.06:04:29
@vcunat:matrix.orgvcunat(we were restarting the queue-runner a lot, and the build takes hours)06:04:40
@leona:leona.isleonaquite a few builds are 'receiving outputs' for multiple hours now: https://hydra.nixos.org/machines on multiple machines08:12:09
@joerg:thalheim.ioMic92I can check again, but yesterday it was genuine compression cpu load it had to do and it was competing with build resources08:28:36
@joerg:thalheim.ioMic92we are using the highest zstd compression ratio08:28:51
@leona:leona.isleona

3h for etc.drv is much..

08:31:57
@leona:leona.isleona * >3h for etc.drv is much.. 08:32:05
@joerg:thalheim.ioMic92Has to wait. I am currently working on something else08:35:19
@joerg:thalheim.ioMic92Deploying to hydra-staging now to sanity check my scheduler chhanges08:52:15
@joerg:thalheim.ioMic92* Deploying to hydra-staging now to sanity check my scheduler changes08:58:32
@hexa:lossy.networkhexa (signing key rotation when)full disk on mimas o_O10:13:14
@hexa:lossy.networkhexa (signing key rotation when) https://grafana.nixos.org/d/rYdddlPWk/node-exporter-full?orgId=1&from=now-2d&to=now&timezone=browser&var-datasource=default&var-job=node&var-node=mimas.nixos.org:9100&var-diskdevices=%5Ba-z%5D%2B%7Cnvme%5B0-9%5D%2Bn%5B0-9%5D%2B%7Cmmcblk%5B0-9%5D%2B&refresh=1m&viewPanel=panel-152 10:13:56
@hexa:lossy.networkhexa (signing key rotation when)what happend last night?10:14:02
@joerg:thalheim.ioMic92[root@mimas:~]# systemctl status nix-gc × nix-gc.service - Nix Garbage Collector Loaded: loaded (/etc/systemd/system/nix-gc.service; linked; preset: ignored) Active: failed (Result: exit-code) since Fri 2026-06-19 09:15:03 UTC; 1h 0min ago Invocation: cde0a9338bf3435c96213d772e8c38b1 TriggeredBy: ● nix-gc.timer Process: 2752969 ExecStart=/nix/store/6j08xq0n9zlz3jdx7fv97svljmkxci0z-unit-script-nix-gc-start/bin/nix-gc-start (code=exited, status=1/FAILURE) Main PID: 2752969 (code=exited, status=1/FAILURE) IP: 0B in, 0B out IO: 300K read, 0B written Mem peak: 8M CPU: 37ms 10:15:49
@joerg:thalheim.ioMic92I restarted it10:16:58
@joerg:thalheim.ioMic92Jun 19 10:16:56 mimas nix-gc-start[2850168]: error: interrupted by the user10:17:08
@joerg:thalheim.ioMic92 hexa (signing key rotation when): ^ was this ue? 10:17:18
@joerg:thalheim.ioMic92 * hexa (signing key rotation when): ^ was this you? 10:17:23
@hexa:lossy.networkhexa (signing key rotation when)yep10:17:23
@hexa:lossy.networkhexa (signing key rotation when)running fast-nix-gc instead10:17:44
@joerg:thalheim.ioMic92alright10:17:59
@hexa:lossy.networkhexa (signing key rotation when)ehhh, the queue-runner came back up10:20:02
@hexa:lossy.networkhexa (signing key rotation when)oh, socket activation10:20:18
@joerg:thalheim.ioMic92oh my god: [root@mimas:~]# du -sh /nix/var/nix/db/db.sqlite 16G /nix/var/nix/db/db.sqlite 10:28:55
@joerg:thalheim.ioMic92I was wondering why nix-fast-gc was taking so long in single-threaded mode10:29:32
@hexa:lossy.networkhexa (signing key rotation when)is this already a size where behaviors get pathological? :P10:29:33
@joerg:thalheim.ioMic92no10:29:41
@joerg:thalheim.ioMic92but you need the RAM10:29:45
@joerg:thalheim.ioMic92I tried to make it efficient10:29:49

Show newer messages


Back to Room ListRoom Version: 6