| 25 Jun 2026 |
 Vladimír Čunát | On retrying I was running into that out-of-space issue with many different builds, as something left 64 GiB build dir and we only have 128 GiB tmpfs on elated-minsky, so we tended to run out with the current combination of build. | 15:35:42 |
| Vladimír Čunát | * On retrying I was running into that out-of-space issue with many different builds, as something left 64 GiB build dir and we only have 128 GiB tmpfs on elated-minsky, so we tended to run out with the current combination of builds. | 15:35:57 |
| Vladimír Čunát | A while ago:
Failed: error: path '/nix/store/36472w3j7c95dvwvxisjawpr14ll9fvf-hunspell-dict-he-il-libreoffice-6.3.0.4' is not valid
https://hydra.nixos.org/build/332729154#tabs-buildsteps (step 7)
| 15:38:22 |
| Vladimír Čunát | So while these issues don't have 100% reproducibility, they do seem to happen relatively often, at least with our high-ish build rate. | 15:39:17 |
| Vladimír Čunát | I'll be trying to keep reasonable balance between amount of failures and amount of restart-all-failed. | 15:40:20 |
 Mic92 | looks like the error that vcunat showed me run before hexa deployed | 15:52:37 |
| Mic92 | this one is after, from the looks of it:
7|1|elated-minsky|1782396035|1782396048|error: path '/nix/store/36472w3j7c95dvwvxisjawpr14ll9fvf-hunspell-dict-he-il-libreoffice-6.3.0.4' is not valid
| 15:55:09 |
 hexa | https://meet.cccda.de/nix-osin-fra in 5m | 15:55:55 |
| Mic92 | Sergei Zimmerman (xokdvium): could it be that we don't have https://github.com/NixOS/nix/pull/15992 and that's why temproots are not respected | 16:00:23 |
| Mic92 | we have concurrency for sure | 16:00:31 |
 Sergei Zimmerman (xokdvium) | In reply to @joerg:thalheim.io
Sergei Zimmerman (xokdvium): could it be that we don't have https://github.com/NixOS/nix/pull/15992 and that's why temproots are not respected Might be. Though I wonder why it wasn’t surfaced before | 16:23:15 |
| Mic92 | Before builds were potentially more serialized over ssh | 16:34:07 |
| Mic92 | * Before builds were potentially more serialized over the SSH worker protocol | 16:34:35 |
| Sergei Zimmerman (xokdvium) | Could you like the relevant pieces of code that does the rooting? One important invariant is that the path should be made a temproot we rely on it being valid | 17:57:02 |
| Sergei Zimmerman (xokdvium) | * Could you like the relevant pieces of code that does the rooting? One important invariant is that the path should be made a temproot before we rely on it being valid | 17:57:17 |
| Sergei Zimmerman (xokdvium) | * Could you link the relevant pieces of code that does the rooting? One important invariant is that the path should be made a temproot before we rely on it being valid | 17:59:17 |
| Mic92 | https://github.com/NixOS/hydra/commit/6735fe6999be400cfb5812316cd994229276f9d6 | 18:00:20 |
| Sergei Zimmerman (xokdvium) | Does it also root already valid paths? | 18:02:45 |
| Sergei Zimmerman (xokdvium) | Hm looks so yeah | 18:03:38 |
| Mic92 | Yes. Right after the pin | 18:04:01 |
| 26 Jun 2026 |
|  Yureka (she/her) joined the room. | 07:33:17 |
| Yureka (she/her) | this job seems stuck in post-processing? https://hydra.nixos.org/build/332754507
or is that just the queue runner having enough other compression to do | 07:33:43 |
| * Yureka (she/her) is gleefully awaiting nixos-unstable bump | 07:34:14 |
 emily | compressing the ISO is probably really slow I guess? | 07:39:00 |
| Yureka (she/her) | the x86 one was done within a minute | 07:43:09 |
| Yureka (she/her) | this one is in post-processing since 4 hours | 07:43:21 |
| Yureka (she/her) | wait... the opposite | 07:44:11 |
| Yureka (she/her) | compare: https://hydra.nixos.org/build/332754492 | 07:44:12 |
| Mic92 | hexa (signing key rotation when): I won't be around much today. I just finished the backport of the nix temproot gc fix and found a small race for uploads on the hydra side: https://github.com/NixOS/infra/pull/1104 | 07:47:19 |
| Mic92 | The nix fix very much looks like what we are seeing in the logs | 07:48:16 |
