| 9 May 2026 |
 K900 | Can we not somehow sandbox them away | 17:15:57 |
 emily | because no FS namespaces on macOS | 17:16:02 |
| K900 | So is the answer just literally no? | 17:16:20 |
| emily | you would have to ban builds when any outputs are present | 17:16:25 |
| emily | which might be okay in practice | 17:16:29 |
| emily | would also likely mean not GCing outputs separately probably | 17:16:45 |
| K900 | For Hydra it's probably fine to just kill the outputs in a prebuild hook or something | 17:16:57 |
| emily | well no since other things could depend on them | 17:17:03 |
| emily | but they should just get substituted | 17:17:07 |
 Randy Eckenrode | Yeah. Found a leaf package that can be used to reproduce.
$ nix build --no-link -f . darwin.dyld
$ nix store delete $(nix eval --raw -f . darwin.dyld.out)
1 store paths deleted, 2.92 MiB freed
$ nix build --no-link -f . darwin.dyld
$ result/bin/dsc_extractor
fish: Job 1, 'result/bin/dsc_extractor' terminated by signal SIGKILL (Forced quit)
| 17:17:09 |
| emily | the question is why that's not happening. it could be an upload-side issue | 17:17:12 |
| emily | I've asked about it in #infra:nixos.org | 17:17:16 |
| Randy Eckenrode | * Yeah. Found a leaf package that can be used to reproduce.
$ nix build --no-link -f . darwin.dyld
$ nix store delete $(nix eval --raw -f . darwin.dyld.out)
1 store paths deleted, 2.92 MiB freed
$ nix build -f . darwin.dyld
$ result/bin/dsc_extractor
fish: Job 1, 'result/bin/dsc_extractor' terminated by signal SIGKILL (Forced quit)
| 17:17:18 |
| emily | because I don't know Hydra well enough to guess at the likely causes | 17:17:22 |
| K900 | I know the outputs are pushed one by one | 17:17:33 |
| emily | right | 17:17:44 |
| emily | but is it really likely for an upload to just die and immediately give up half-way through outputs? | 17:17:52 |
| emily | sounds weird to me, but I don't know | 17:17:59 |
| emily | hypotheses that involve "Hydra's connection to S3 is just weirdly flaky" don't immediately strike me as plausible which is why I thought the whole thing was probably a hallucinated red herring going in | 17:18:27 |
| K900 | No but I wonder if it's possible we're firing off a build before all the outputs are fully done and committed | 17:18:32 |
| emily | oh, yeah | 17:18:43 |
| emily | the race condition seems plausible since we do get multiple builds happening on Hydra, I did mention that in an earlier comment | 17:18:57 |
| emily | but I guess forgot :) | 17:19:01 |
| Randy Eckenrode | This reproduces from master.
$ nix shell github:NixOS/nixpkgs#darwin.dyld --command dsc_extractor
$ nix develop github:NixOS/nixpkgs#darwgithub:NixOS/nixpkgs#darwin.dyld —command dsc_extractor/nixpkgs#darwin.dyld.out)
$ nix shell --option substitute false github:NixOS/nixpkgs#darwin.dyld —command dsc_extractor
| 17:24:11 |
| Randy Eckenrode | * This reproduces from master.
$ nix shell github:NixOS/nixpkgs#darwin.dyld --command dsc_extractor
$ nix store delete $(nix eval --raw github:NixOS/nixpkgs#darwin.dyld.out)
$ nix develop github:NixOS/nixpkgs#darwgithub:NixOS/nixpkgs#darwin.dyld —command dsc_extractor/nixpkgs#darwin.dyld.out)
$ nix shell --option substitute false github:NixOS/nixpkgs#darwin.dyld —command dsc_extractor
| 17:24:40 |
| Randy Eckenrode | * This reproduces from master.
$ nix shell github:NixOS/nixpkgs#darwin.dyld --command dsc_extractor
$ nix store delete $(nix eval --raw github:NixOS/nixpkgs#darwin.dyld.out)
$ nix develop github:NixOS/nixpkgs#darwgithub:NixOS/nixpkgs#darwin.dyld —command dsc_extractor
$ nix shell --option substitute false github:NixOS/nixpkgs#darwin.dyld —command dsc_extractor
| 17:25:00 |
| emily | yeah, I mean it's unsurprising that this does break things, it's the same as --rebuild | 17:25:03 |
| emily | what surprises me is that there's at least one piece of evidence that it's what's causing the persistent Hydra issues | 17:25:14 |
| Randy Eckenrode | * This reproduces from master.
$ nix shell github:NixOS/nixpkgs#darwin.dyld --command dsc_extractor
$ nix store delete $(nix eval --raw github:NixOS/nixpkgs#darwin.dyld.out)
$ nix develop github:NixOS/nixpkgs#darwgithub:NixOS/nixpkgs#darwin.dyld --command dsc_extractor
$ nix shell --option substitute false github:NixOS/nixpkgs#darwin.dyld --command dsc_extractor
| 17:25:18 |
| emily | given ^, it didn't seem like there would be common cases where Hydra would be building only some outputs | 17:25:32 |
