| 24 Jun 2026 |
 dramforever | builder gc race, generally known | 20:17:57 |
| 25 Jun 2026 |
|  unclechu joined the room. | 03:27:22 |
| unclechu | Hi, I was adviced to report it here too. Got this nixos-rebuild failure:
warning: unable to download 'https://cache.nixos.org/nar/0wj3h178aczc4wfhm0c5pzzzv1vv9nlkbvcar8ibhk82jl4gjlf8.nar.zst': HTTP error 200 (curl error: Stream error in the HTTP/2 framing layer); retrying from offset 5242880 in 303 ms (attempt 1/5)
warning: unable to download 'https://cache.nixos.org/nar/0wj3h178aczc4wfhm0c5pzzzv1vv9nlkbvcar8ibhk82jl4gjlf8.nar.zst': HTTP error 206 (curl error: Stream error in the HTTP/2 framing layer); retrying from offset 6291456 in 546 ms (attempt 2/5)
warning: unable to download 'https://cache.nixos.org/nar/0wj3h178aczc4wfhm0c5pzzzv1vv9nlkbvcar8ibhk82jl4gjlf8.nar.zst': HTTP error 206 (curl error: Stream error in the HTTP/2 framing layer); retrying from offset 7340032 in 1013 ms (attempt 3/5)
warning: unable to download 'https://cache.nixos.org/nar/0wj3h178aczc4wfhm0c5pzzzv1vv9nlkbvcar8ibhk82jl4gjlf8.nar.zst': HTTP error 206 (curl error: Stream error in the HTTP/2 framing layer); retrying from offset 8388608 in 2306 ms (attempt 4/5)
building '/nix/store/nnr93pkbh92g4ww3c0djigch3r8bviz9-etc-UPower-UPower.conf.drv'...
building '/nix/store/nny85ggld2jazzzn13rawqps3sfqnc02-etc-X11-xorg.conf.d-00-keyboard.conf.drv'...
building '/nix/store/571rz6khvha6yszmxkqphjnlj7y6cghq-etc-bash_logout.drv'...
building '/nix/store/6k8w1wb91jndyz93lirg1cpcgqmdxf15-etc-environment.d-50-systemd-path.conf.drv'...
error: failed to read compressed data (Zstd decompression failed: Data corruption detected)
error: path '/nix/store/a9j9wjmhm9v6mp836iwa4ayzwv99iiwd-bibata-cursors-2.0.7' is required, but there is no substituter that can build it
| 03:28:37 |
| unclechu | I retried and got the same result. After waiting for few minutes it seems the issue is not reproducing anymore. | 03:29:05 |
| dramforever | yeah this is a known issue https://github.com/NixOS/infra/pull/1101 | 04:26:27 |
| dramforever | not sure if it's supposed to be fixed on hydra tho | 04:26:33 |
|  Nick joined the room. | 04:58:18 |
 Mic92 | Looks because of some rebase shenanigans, I overwrote one of my changes. In the older version I had these operations swapped and also my gcroot test was no longer in the tree
if pin_path(roots, &path).await && local_db.is_valid_path(&path).await? {
This would explain why we still see those gcroot errors.
hexa (signing key rotation when) would you be around for enough deploy round?
| 06:56:39 |
 Grimmauld (any/all) | error (ignored): error: failed to read compressed data (Truncated zstd input)
Oh look!
Anything i can/should do about it? | 06:57:19 |
| Mic92 | so: curl -v https://cache.nixos.org/nar/0wj3h178aczc4wfhm0c5pzzzv1vv9nlkbvcar8ibhk82jl4gjlf8.nar.zst
gives me < last-modified: Wed, 24 Jun 2026 04:09:41 GMT | 06:58:40 |
| Mic92 | Was this before or after we deployed the nar upload race? | 06:58:51 |
| Mic92 | https://github.com/NixOS/infra/pull/1101 | 06:59:53 |
 Vladimír Čunát | Looks like before deploying that fix. | 07:04:19 |
| Vladimír Čunát | Before you even opened that PR. | 07:04:45 |
| Mic92 | Getting all the nars would take a bit. we could flush the cache? | 07:06:43 |
| Mic92 | * Getting all the nars would take a bit. we could flush the fastly cache? | 07:06:51 |
| Mic92 | * Getting all the nars would take a bit looking at my last attempt. We could flush the fastly cache? | 07:07:06 |
| Grimmauld (any/all) | nevermind, i just got scared, but the underlying failure was my fault and this was just a symptom | 07:15:14 |
| Mic92 | vcunat: do you know what the TTL actually is at the moment? | 07:16:22 |
| Vladimír Čunát | 24h if I read it correctly. | 07:16:36 |
| Mic92 | Okay. Hexa merged 18h ago | 07:17:33 |
| Mic92 | We could wait 6 more hours? | 07:17:46 |
| Mic92 | Presumably a large part of paths already expired before that. | 07:18:26 |
 hexa | I am now | 08:14:49 |
|  Yorusaka Miyabi joined the room. | 09:05:33 |
| Mic92 | * Looks because of some rebase shenanigans, I overwrote one of my changes. In the older version I had these operations swapped and also my gcroot test was no longer in the tree
if pin_path(roots, &path).await && local_db.is_valid_path(&path).await? {
This would explain why we still see those gcroot errors.
hexa (signing key rotation when) would you be around for another deploy round?
| 09:21:31 |
| Mic92 | hexa (signing key rotation when): okay. I got this branch: https://github.com/NixOS/infra/pull/1102 | 09:26:10 |
| Mic92 | also builder | 09:26:30 |
| hexa | ok, in a bit | 09:35:11 |
| hexa | doing | 10:08:06 |
