| 7 Oct 2024 |
 emily | personally, I don't think that objection makes sense if we're working towards switching to the minimal bootstrap by default | 14:03:25 |
| emily | which itself involves absurd grand tours of decades of software history | 14:03:36 |
| *  raboof dreams about ca-derivations where the 'long-way' and 'shortcut' builds arrive at the same result, but that's probably too much work to be feasible :) ) | 14:06:37 |
| emily | I think as long as our rustc is reproducible that should already be the case | 14:07:55 |
| emily | it's more a matter of making contributors and Hydra do the whole rigmarole on staging | 14:08:08 |
| emily | anyway, hopefully we get an independent Rust implementation that can bootstrap rustc without a long chain at some point. e.g. https://notgull.net/announcing-dozer/ or something | 14:08:34 |
| emily | but I'd really like to see a source-based bootstrap, personally | 14:08:49 |
|  Sam Lehman changed their profile picture. | 14:24:49 |
| 8 Oct 2024 |
|  kait joined the room. | 11:00:40 |
| 9 Oct 2024 |
|  chayleaf joined the room. | 11:35:31 |
| chayleaf | can someone please look into https://github.com/NixOS/nixpkgs/issues/346419? the binary cache is missing a file compared to local build for some reason | 12:26:01 |
| raboof | interesting. testing with nix-build '<nixpkgs>' -A freeplane --check --keep-failed it seems to consistently not generate that file - do you think the nondeterminism is in the 'main' build or one of its inputs? | 12:35:44 |
| chayleaf | the inputs are all cached, so its definitely the main build | 12:38:30 |
| chayleaf | basically:
- i haven't built any of the native dependencies myself, they are copied from the cache
- the java dependencies from gradle are downloaded by url+hash pairs, so they can't really differ
| 12:44:20 |
| raboof | tbh I don't see anything obvious in https://github.com/search?q=repo%3Afreeplane%2Ffreeplane%20globalBin&type=code that'd put that file there... maybe compare the builds logs between your local build and the one from the binary cache? | 12:55:56 |
| chayleaf | the tasks run in a different order, maybe parallel building is the issue? | 14:16:40 |
| chayleaf | if its consistently not generating that file for you, can you try setting enableParallelBuilding to false? | 14:17:18 |
| raboof | then I indeed see it | 14:21:10 |
| 12 Oct 2024 |
|  steeringwheelrules joined the room. | 22:47:32 |
| 14 Oct 2024 |
 Moritz Sanft | Hey, we think there's a substantial problem about Nix's store optimisations in conjunction with reproducible builds, and we'd like to hear some opinions on it.
Consider the following:
2 distinct Files in Nix store, same content (-> same NAR hash). But different permissions, except for executability (e.g. 600 and 644). NAR hash is still the same. nix store optimise looks for deduplication possibilities by matching on NAR hashes. Finds the 2 files and hardlinks one to the other.
File is used in an archive, image, etc. (i.e. any file where the permissions influence its actual contents) - Now this file is inherently irreproducible.
I think that either Nix should not use NAR hashes for store optimisation, change NAR to reflect all permissions (which is unrealistic due to the impact of the change). But curious to hear what you think about this. We just tracked down a image reproducibility bug we fought with for a long time to this and it was very annoying to actually find this. | 08:16:40 |
| Moritz Sanft | CC @Paul Meyer (katexochen) | 08:16:50 |
 Sandro 🐧 | Isn't this a common thing that such files need to resolve hard links? | 15:47:15 |
| Sandro 🐧 | I think we had such problems before, not necessarily with permissions though | 15:47:34 |
| raboof | shouldn't files in the store typically have 444/555 permissions anyway? or is that different for hardlinks? | 15:49:47 |
 Paul Meyer (katexochen) | They should, at least this looks like it: https://github.com/NixOS/nix/blob/d5c45952acffebce29873f14e5eeca3ac78cbe26/src/libstore/posix-fs-canonicalise.hh#L23 | 16:18:19 |
| Paul Meyer (katexochen) | Grepping nix store for writable files returns a ton of writable files, both on my local system as well as on my remote builder:
find /nix/store -type f -perm -u+w ! -perm -g+w ! -perm -o+w
Those have also modified time stamps. 🤔 | 16:20:52 |
| Paul Meyer (katexochen) | * Grepping nix store for writable files returns a ton, both on my local system as well as on my remote builder:
find /nix/store -type f -perm -u+w ! -perm -g+w ! -perm -o+w
Those have also modified time stamps. 🤔 | 16:21:07 |
| Paul Meyer (katexochen) | stat /nix/store/a3b4xpykc7x1gz7d5kvcw40qf0xlqg4n-python3.11-fabric-3.2.2/lib/python3.11/site-packages/fabric-3.2.2.dist-info/top_level.txt
File: /nix/store/a3b4xpykc7x1gz7d5kvcw40qf0xlqg4n-python3.11-fabric-3.2.2/lib/python3.11/site-packages/fabric-3.2.2.dist-info/top_level.txt
Size: 7 Blocks: 8 IO Block: 4096 regular file
Device: 0,41 Inode: 189116731 Links: 1
Access: (0644/-rw-r--r--) Uid: (872415232/ UNKNOWN) Gid: (872415232/ UNKNOWN)
Access: 2024-10-14 12:11:20.405597533 +0000
Modify: 2024-10-14 12:11:20.405597533 +0000
Change: 2024-10-14 12:11:20.444598825 +0000
Birth: 2024-10-14 12:11:20.405597533 +0000
| 16:23:18 |
| Paul Meyer (katexochen) | I can reproduce this using nixpkgs-review + remote builder with a commit that doesn't build. For example, running nixpkgs-review rev HEAD on https://github.com/katexochen/nixpkgs/pull/new/repro/azure-cli/2-65 will lead to the following on the configured remote builder:
$ ls -la /nix/store/*azure-cli* | grep nixbld
drwxrwxr-t 1 root nixbld 6669116 Oct 14 18:01 ..
drwxrwxr-t 1 root nixbld 6669116 Oct 14 18:01 ..
drwxrwxr-t 1 root nixbld 6669116 Oct 14 18:01 ..
drwxrwxr-t 1 root nixbld 6669116 Oct 14 18:01 ..
drwxrwxr-t 1 root nixbld 6669116 Oct 14 18:01 ..
| 18:12:51 |
| Paul Meyer (katexochen) | * I can reproduce this using nixpkgs-review + remote builder with a commit that doesn't build. For example, running nixpkgs-review rev HEAD on https://github.com/katexochen/nixpkgs/tree/repro/azure-cli/2-65 will lead to the following on the configured remote builder:
$ ls -la /nix/store/*azure-cli* | grep nixbld
drwxrwxr-t 1 root nixbld 6669116 Oct 14 18:01 ..
drwxrwxr-t 1 root nixbld 6669116 Oct 14 18:01 ..
drwxrwxr-t 1 root nixbld 6669116 Oct 14 18:01 ..
drwxrwxr-t 1 root nixbld 6669116 Oct 14 18:01 ..
drwxrwxr-t 1 root nixbld 6669116 Oct 14 18:01 ..
| 18:13:49 |
