!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

426 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.131 Servers

Load older messages


SenderMessageTime
10 Jun 2026
@k900:0upti.meK900Cancelled nixos-unstable{,-small} from 5 minutes ago06:30:08
@k900:0upti.meK900 For https://github.com/NixOS/nixpkgs/pull/530238 06:30:13
@k900:0upti.meK900Will restart once that lands06:30:28
@k900:0upti.meK900Ughhh06:54:58
@k900:0upti.meK900 @vcunat can you cancel the evals 06:55:01
@vcunat:matrix.orgVladimír ČunátI already did.06:56:42
@k900:0upti.meK900Thanks06:58:14
@vcunat:matrix.orgVladimír ČunátI haven't confirmed any failure in the Hydra's UI since then. I looked specifically at ISO builds, as those seemed to suffer abnormally from this. I don't know if this evidence has statistical significance already, though.14:34:52
@vcunat:matrix.orgVladimír ČunátLooking at hydra-queue-runner logs, we have lots of cases even today where the attempt counter goes over 10/32.14:37:32
@vcunat:matrix.orgVladimír ČunátWait... "retrying in 2346603 ms". Thats 39 minutes. 14:39:32
@vcunat:matrix.orgVladimír Čunát* Wait... "retrying in 2346603 ms". That's 39 minutes. 14:39:42
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)Mhm yeah I mentioned that we should cap the retry time limit, I think that on master but not 2.3414:40:30
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)Or even just make these failures exempt, those don’t need exponential back off at all14:41:06
@vcunat:matrix.orgVladimír ČunátIf the cap is 32 attempts, the backoff goes up to 2^32? 🤣14:42:21
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)😵‍💫14:42:33
@hexa:lossy.networkhexawelp14:57:14
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)Lemme add a no delay handling for these sorts of errors15:02:13
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)https://github.com/aws/aws-cli/issues/401#issuecomment-88380435915:02:22
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)On 2.34 it would overflow apparently15:16:24
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)Well, if it had the chance to that is15:16:32
@xokdvium:matrix.orgSergei Zimmerman (xokdvium) vcunat: I've pushed this https://github.com/NixOS/nix/pull/15949/changes/15ee316650a282432676207e472de8d2f7a77f6c.
I hate everything about this, but such is AWS apparently?
15:19:43
@hexa:lossy.networkhexaloving it15:20:38
11 Jun 2026
@hexa:lossy.networkhexahttps://meet.cccda.de/nix-osin-fra16:00:54
@jfly:matrix.orgJeremy Fleischman (jfly) Sorry to miss, travelling this week. 16:34:36
22 May 2021
@grahamc:nixos.org@grahamc:nixos.org set the history visibility to "world_readable".17:01:28
@grahamc:nixos.org@grahamc:nixos.org changed the room name to "" from "".17:01:28
@cole-h:matrix.orgcole-h joined the room.17:03:05
@andi:kack.itandi- joined the room.17:18:59
@sandro:supersandro.deSandro joined the room.17:21:35
@hexa:lossy.networkhexa joined the room.17:22:33

Show newer messages


Back to Room ListRoom Version: 6