| 19 Feb 2025 |
 emily | that's how it works in Nix/Nixpkgs; Hydra won't build things with non-free licences, so they are never substituted from the default substituters configuration, but nothing prevents you from caching them yourself | 02:57:10 |
 apteryx | you may still want CI to be able to build something to detect problems, but not distributing it as a substitute | 02:59:05 |
| apteryx | or in the case of the zfs combination with linux in an initrd, a server could have cached such initrd because it's using zfs itself for its file system, etc. | 02:59:43 |
| apteryx | so it seems to have value to be able to tag a package with 'this is for building locally only, in all situations' | 03:00:10 |
| emily | fair enough, if you have logic to avoid pushing !allowSubstitutes derivations to a cache etc. | 03:00:37 |
| emily | though I still think you only need it on the "push" end, not the "pull", where the issues of the mechanism arise :) | 03:00:48 |
| apteryx | at least with the way things are currently working with Guix (and I assume, Nix), anything in the store is available as a substitute, so there's no fine control on the push. | 03:01:48 |
| apteryx | if I'm not mistaken | 03:02:23 |
| emily | right, so I don't understand how allowSubstitutes helps… anyone who knows the hash could download the illegal binary from the cache, even if your client avoids doing so by default, which seems bad | 03:02:47 |
| apteryx | good point | 03:03:51 |
| apteryx | I'll review that part of the infra (how /gnu/store things end up being served as nars) on the substitute servers. Perhaps I can add some logic to prevent things marked as non-substitutable ending up as a .nar ready to be served. | 03:06:44 |
| apteryx | thanks for the ideas! | 03:08:01 |
| emily | I feel it's the wrong mechanism for this still, but ok :) | 03:15:09 |
| apteryx | For you the best option would be to not build such binary at all? Or something else? | 03:19:44 |
| apteryx | or perhaps we need a #:distributable? argument, orthogonal of substitutability | 03:20:31 |
| emily | any mechanism about not distributing a problematic derivation output is wholly orthogonal to whether the client can try substituting it, which there's no reason to let a derivation block as there are always legitimate use-cases (private LAN etc.) and which forbidding gets in the way of ("can't cache a closure properly because one of them is marked as !allowSubstitutes because it's meant to be trivial so build inputs get pulled in anyway") | 03:21:38 |
| emily | as in, what allowSubstitutes is meant to do doesn't help solve the problem in question in any way, and what it's meant to do has largely (on the Nix side at least) turned out to hurt more than it helps | 03:22:14 |
| emily | (I'd be hesitant to have CI build stuff that's considered legally problematic enough to not be distributable in the first place, but I guess Guix is strict enough about licensing that something like zfs.ko is probably the limit of the risk there.) | 03:23:28 |
 mra | In reply to @emilazy:matrix.org
as in, what allowSubstitutes is meant to do doesn't help solve the problem in question in any way, and what it's meant to do has largely (on the Nix side at least) turned out to hurt more than it helps one question: another use of allowSubstitutes = 0 on Guix is for HPC packages, specifically those which have CPU-specific optimisations, so that a client doesn't substitute a package which is optimised for a different CPU. How does Nix handle this case? | 06:54:40 |
| mra | This is somewhat orthogonal to the distributability concern, I agree, but this is currently one of the main applications of non-substitutability for us. | 06:55:50 |
 Eli Kogan-Wang | In reply to @morgan.arnold:matrix.org
one question: another use of allowSubstitutes = 0 on Guix is for HPC packages, specifically those which have CPU-specific optimisations, so that a client doesn't substitute a package which is optimised for a different CPU. How does Nix handle this case? Are you asking about https://wiki.nixos.org/wiki/Build_flags?
See https://github.com/NixOS/nixpkgs/pull/202526#issue-1461820752 | 07:21:01 |
| mra | Oh, interesting. It just has to be specifically requested. That makes sense. | 07:30:28 |
| emily | In reply to @morgan.arnold:matrix.org
one question: another use of allowSubstitutes = 0 on Guix is for HPC packages, specifically those which have CPU-specific optimisations, so that a client doesn't substitute a package which is optimised for a different CPU. How does Nix handle this case? I don't understand how this would ever arise. different flags would mean different derivation hashes so you'd never get an incorrect substitution, right? | 14:12:12 |
| emily | if you mean using -march=native to get an impure build, I'd suggest just not doing that. it's cheap to specify the relevant platform explicitly and fixes the determinism issue | 14:12:57 |
| emily | FYI, the 2.26 update breaks buildInputs = [ nixVersions.nix_2_26 ]; | 23:03:20 |
| emily | it has .dev and .libs (should be .lib?) attributes in passthru, but those are not proper outputs | 23:03:41 |
| emily | uh, and .dev is just … empty | 23:04:21 |
| emily | ok I guess you have to use .dev.dev. anyway this is very weird and breaking. | 23:05:16 |
 ElvishJerricco | emily: yes, 2.26 is now componentized and the nix_2_26 build is basically just a symlink farm of all the components | 23:27:18 |
| ElvishJerricco | you have to depend on the specific libs you need via the libs passthru I think | 23:27:53 |
