 | NixOS Reproducible Builds | 461 Members |
| Report: https://reproducible.nixos.org Project progress: https://github.com/orgs/NixOS/projects/30 | 97 Servers |
| NixOS Reproducible Builds | 461 Members |
| Report: https://reproducible.nixos.org Project progress: https://github.com/orgs/NixOS/projects/30 | 97 Servers |
| Sender | Message | Time |
|---|---|---|
| 8 Oct 2024 | ||
 kait joined the room. | 11:00:40 | |
| 9 Oct 2024 | ||
 chayleaf joined the room. | 11:35:31 | |
| chayleaf | can someone please look into https://github.com/NixOS/nixpkgs/issues/346419? the binary cache is missing a file compared to local build for some reason | 12:26:01 |
 raboof | interesting. testing with nix-build '<nixpkgs>' -A freeplane --check --keep-failed it seems to consistently not generate that file - do you think the nondeterminism is in the 'main' build or one of its inputs? | 12:35:44 |
| chayleaf | the inputs are all cached, so its definitely the main build | 12:38:30 |
| chayleaf | basically:
| 12:44:20 |
| raboof | tbh I don't see anything obvious in https://github.com/search?q=repo%3Afreeplane%2Ffreeplane%20globalBin&type=code that'd put that file there... maybe compare the builds logs between your local build and the one from the binary cache? | 12:55:56 |
| chayleaf | the tasks run in a different order, maybe parallel building is the issue? | 14:16:40 |
| chayleaf | if its consistently not generating that file for you, can you try setting enableParallelBuilding to false? | 14:17:18 |
| raboof | then I indeed see it | 14:21:10 |
| 12 Oct 2024 | ||
 steeringwheelrules joined the room. | 22:47:32 | |
| 14 Oct 2024 | ||
 Moritz Sanft | Hey, we think there's a substantial problem about Nix's store optimisations in conjunction with reproducible builds, and we'd like to hear some opinions on it. Consider the following:
I think that either Nix should not use NAR hashes for store optimisation, change NAR to reflect all permissions (which is unrealistic due to the impact of the change). But curious to hear what you think about this. We just tracked down a image reproducibility bug we fought with for a long time to this and it was very annoying to actually find this. | 08:16:40 |
| Moritz Sanft | CC @Paul Meyer (katexochen) | 08:16:50 |
 Sandro 🐧 | Isn't this a common thing that such files need to resolve hard links? | 15:47:15 |
| Sandro 🐧 | I think we had such problems before, not necessarily with permissions though | 15:47:34 |
| raboof | shouldn't files in the store typically have 444/555 permissions anyway? or is that different for hardlinks? | 15:49:47 |
 Paul Meyer (katexochen) | They should, at least this looks like it: https://github.com/NixOS/nix/blob/d5c45952acffebce29873f14e5eeca3ac78cbe26/src/libstore/posix-fs-canonicalise.hh#L23 | 16:18:19 |
| Paul Meyer (katexochen) | Grepping nix store for writable files returns a ton of writable files, both on my local system as well as on my remote builder:find /nix/store -type f -perm -u+w ! -perm -g+w ! -perm -o+wThose have also modified time stamps. 🤔 | 16:20:52 |
| Paul Meyer (katexochen) | * Grepping nix store for writable files returns a ton, both on my local system as well as on my remote builder:find /nix/store -type f -perm -u+w ! -perm -g+w ! -perm -o+wThose have also modified time stamps. 🤔 | 16:21:07 |
| Paul Meyer (katexochen) | | 16:23:18 |
| Paul Meyer (katexochen) | I can reproduce this using nixpkgs-review + remote builder with a commit that doesn't build. For example, running | 18:12:51 |
| Paul Meyer (katexochen) | * I can reproduce this using nixpkgs-review + remote builder with a commit that doesn't build. For example, running | 18:13:49 |
| 15 Oct 2024 | ||
 Atemu | In reply to @msanft:matrix.orgI don't really see that as an r13y bug, as the actual bug is that a NAR-relevant modification of the store path is performed with a file that was -x before becoming +x (or vice versa? That'd be even worse.) | 04:14:54 |
| Atemu | Paul Meyer (katexochen): I think that's an existing bug where some programs ran as root are somehow able to remount the Nix store read-write and other programs are then able to modify their store paths which some of them do. | 04:16:15 |
| raboof | In reply to @katexochen:matrix.orgright, so those files can have write permissions when they've been written to the store but then the build fails before the files were 'canonicalised'. If the build fails before completing then the derivation should not have been added to the store db, and thus should not be used by any other derivation yet, though, right? so that doesn't seem like it explains the problem you and Moritz Sanft saw yet? | 07:02:19 |
| raboof | In reply to @katexochen:matrix.orginterestingly this returns no results on my NixOS installation (which doesn't use store optimization). Are those files hardlinked by store optimization? | 07:05:06 |
 emily | oh, so even Linux users have optimization-based store corruption! | 07:05:14 |
| raboof | In reply to @katexochen:matrix.org* right, so those files can have write permissions when they've been written to the store but then the build fails before the files were 'canonicalised'? If the build fails before completing then the derivation should not have been added to the store db, and thus should not be used by any other derivation yet, though, right? so that doesn't seem like it explains the problem you and Moritz Sanft saw yet? | 07:06:07 |
| Paul Meyer (katexochen) | In reply to @raboof:matrix.org The one from above isn't: | 07:09:42 |
| Paul Meyer (katexochen) | No explanation how the writable file ended up in the image, no. I'd have to look deeper into the hardlink optimization. Maybe there is a race with creating the link under /nix/store/.links and the build failing? Then any file from a successful derivation could be hardlinked against a writable file. | 07:15:35 |