| 10 Jun 2026 |
 K900 | Thanks | 06:58:14 |
 vcunat | I haven't confirmed any failure in the Hydra's UI since then. I looked specifically at ISO builds, as those seemed to suffer abnormally from this. I don't know if this evidence has statistical significance already, though. | 14:34:52 |
| vcunat | Looking at hydra-queue-runner logs, we have lots of cases even today where the attempt counter goes over 10/32. | 14:37:32 |
| vcunat | Wait... "retrying in 2346603 ms". Thats 39 minutes. | 14:39:32 |
| vcunat | * Wait... "retrying in 2346603 ms". That's 39 minutes. | 14:39:42 |
 Sergei Zimmerman (xokdvium) | Mhm yeah I mentioned that we should cap the retry time limit, I think that on master but not 2.34 | 14:40:30 |
| Sergei Zimmerman (xokdvium) | Or even just make these failures exempt, those don’t need exponential back off at all | 14:41:06 |
| vcunat | If the cap is 32 attempts, the backoff goes up to 2^32? 🤣 | 14:42:21 |
| Sergei Zimmerman (xokdvium) | 😵💫 | 14:42:33 |
 hexa (signing key rotation when) | welp | 14:57:14 |
| Sergei Zimmerman (xokdvium) | Lemme add a no delay handling for these sorts of errors | 15:02:13 |
| Sergei Zimmerman (xokdvium) | https://github.com/aws/aws-cli/issues/401#issuecomment-883804359 | 15:02:22 |
| Sergei Zimmerman (xokdvium) | On 2.34 it would overflow apparently | 15:16:24 |
| Sergei Zimmerman (xokdvium) | Well, if it had the chance to that is | 15:16:32 |
| Sergei Zimmerman (xokdvium) | vcunat: I've pushed this https://github.com/NixOS/nix/pull/15949/changes/15ee316650a282432676207e472de8d2f7a77f6c.
I hate everything about this, but such is AWS apparently? | 15:19:43 |
| hexa (signing key rotation when) | loving it | 15:20:38 |
