!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

482 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.151 Servers

Load older messages


SenderMessageTime
27 Jun 2026
@hexa:lossy.networkhexabumped14:18:34
@k900:0upti.meK900If we have to deal with this the entire time14:18:46
@hexa:lossy.networkhexahttps://github.com/NixOS/infra/pull/1104 was supposed to resolve that iirc14:19:15
@emilazy:matrix.orgemilywhy is it only happening on Darwin(?)14:21:27
@k900:0upti.meK900It's not14:21:49
@k900:0upti.meK900But the darwins are slower so more likely to hit it14:21:56
@emilazy:matrix.orgemily

the darwins

you mean the Darwin

14:22:22
@hexa:lossy.networkhexagc is hourly on linux14:22:43
@hexa:lossy.networkhexaand twice per hour on darwin14:22:47
@hexa:lossy.networkhexa🤡14:22:50
@emilazy:matrix.orgemilydo we really need to GC that often?14:23:24
@hexa:lossy.networkhexawe can try and reduce14:23:56
@k900:0upti.meK900I assume those have less disk also?14:23:57
@emilazy:matrix.orgemilyoh I just mean in general14:24:03
@hexa:lossy.networkhexabut fixing gc was annoying for the small macs with little disk14:24:08
@hexa:lossy.networkhexathe linux builders have between 2x2 and 2x4TB NVME14:24:46
@hexa:lossy.networkhexathe macs have like 256 for the hetzner ones14:24:58
@hexa:lossy.networkhexaand more for the oakhost ones14:25:04
@hexa:lossy.networkhexawe can try fast-nix-gc14:26:19
@hexa:lossy.networkhexawhat would it fix? I guess gcing faster would be benefitial to the nodes anyhow14:26:59
@hexa:lossy.networkhexabut the window during which we'd gc would become smaller14:27:10
@emilazy:matrix.orgemilyyeah I'm just surprised that even Linux builders fill up that quickly14:27:17
@emilazy:matrix.orgemilydoing it twice an hour would be annoying if you wanted to drain the builders presumably14:27:38
@hexa:lossy.networkhexaI don't think we tuned them as much14:27:41
@hexa:lossy.networkhexawe like to keep them fuller so they have deps they might need later14:27:59
@emilazy:matrix.orgemilywonder how easy it is to just make the builds take the GC lock14:28:01
@hexa:lossy.networkhexaand with fast-nix-gc we can tell it to keep newer things alive14:28:17
@emilazy:matrix.orgemily Sergei Zimmerman (xokdvium): how easy would it be to have an option for "refuse to try building something if some of its outputs are present" that defaults on if fallback paths would be used (i.e. macOS, unsandboxed Linux) and then we could set it on for all of Hydra? 14:31:17
@emilazy:matrix.orgemilyI guess that still doesn't handle the case where some of the outputs reached the cache but others didn't, and another machine tries doing the build to satisfy another output, but hasn't substituted the outputs that are in the cache14:31:40
@emilazy:matrix.orgemilythen the cache still gets frankenbuilds14:31:46

Show newer messages


Back to Room ListRoom Version: 6