| 19 Feb 2025 |
 apteryx | any idea how I can pretty print a nix::Goal shared pointer in GDB? | 02:16:03 |
| apteryx | a std::set containing such objects, I meant | 02:16:21 |
| apteryx | p goals looks like this: $5 = std::set with 1 element = {[0] = std::shared_ptr<nix::Goal> (use count 1, weak count 3) = {get() = 0x72d570}} | 02:20:35 |
| apteryx | starting to look like the problem is that buildPaths builds a lists of substitutable goals, which include the output of an unsubstitutable derivation: | 02:32:06 |
| apteryx | 2: *goal = {<std::enable_shared_from_this<nix::Goal>> = {_M_weak_this = std::weak_ptr<nix::Goal> (use count 2, weak count 3) = {get() = 0x5e29a0}}, _vptr.Goal = 0x4c75e0 <vtable for nix::SubstitutionGoal+16>, worker = @0x7fffffff85c0, waitees = std::set with 0 elements, waiters = std::__cxx11::list = {[0] = std::weak_ptr<nix::Goal> (use count 1, weak count 5) = {get() = 0x5b08f0}}, nrFailed = 0, nrNoSubstituters = 0, nrIncompleteClosure = 0, name = "substitution of `/gnu/store/b5sn5ha961hab37r7vl5p2n6sf46x582-texlivetexmf-20240312'", exitCode = nix::Goal::ecBusy}
(gdb) where
#0 nix::Worker::run (this=0x7fffffff85c0, _topGoals=std::set with 1 element = {...}) at nix/libstore/build.cc:3448
#1 0x00000000004659c9 in nix::LocalStore::buildPaths (this=0x55b4d0, drvPaths=std::set with 1 element = {...}, buildMode=nix::bmNormal) at nix/libstore/build.cc:3642
#2 0x000000000040ad6c in performOp (trusted=false, clientVersion=356, from=..., to=..., op=9) at nix/nix-daemon/nix-daemon.cc:481
#3 0x000000000040e01b in processConnection (trusted=false, userId=1000) at nix/nix-daemon/nix-daemon.cc:841
#4 0x000000000040e754 in operator() (__closure=0x524f50) at nix/nix-daemon/nix-daemon.cc:1003
#5 0x0000000000410fda in std::__invoke_impl<void, acceptConnection(int)::<lambda()>&>(std::__invoke_other, struct {...} &) (__f=...) at /gnu/store/d50i890p2lg97kvc131p62wy52krapbd-profile/include/c++/bits/invoke.h:61
#6 0x000000000040f718 in std::__invoke_r<void, acceptConnection(int)::<lambda()>&>(struct {...} &) (__fn=...) at /gnu/store/d50i890p2lg97kvc131p62wy52krapbd-profile/include/c++/bits/invoke.h:154
#7 0x000000000040f5db in std::_Function_handler<void(), acceptConnection(int)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /gnu/store/d50i890p2lg97kvc131p62wy52krapbd-profile/include/c++/bits/std_function.h:290
#8 0x0000000000441f00 in std::function<void()>::operator() (this=0x7fffffffaf10) at /gnu/store/d50i890p2lg97kvc131p62wy52krapbd-profile/include/c++/bits/std_function.h:590
#9 0x00000000004ab2ef in nix::startProcess (fun=..., dieWithParent=false, errorPrefix="unexpected build daemon error: ", runExitHandlers=true) at nix/libutil/util.cc:1025
#10 0x000000000040ed6b in acceptConnection (fdSocket=3) at nix/nix-daemon/nix-daemon.cc:977
#11 0x000000000040f40e in daemonLoop (sockets=std::vector of length 1, capacity 1 = {...}) at nix/nix-daemon/nix-daemon.cc:1055
#12 0x000000000040f4bd in run (sockets=std::vector of length 1, capacity 1 = {...}) at nix/nix-daemon/nix-daemon.cc:1064
#13 0x000000000041a262 in main (argc=12, argv=0x7fffffffc208) at nix/nix-daemon/guix-daemon.cc:569
| 02:32:14 |
| apteryx | texlivetexmf is a private package marked with #:substitutable? #f while texlive uses texlivetexmf as an input but isn't marked itself as non-substitutable. | 02:34:11 |
 emily | I think you want to control such things on the "production" end (i.e. CI doesn't build such binaries) rather than "consumption" (i.e. can you fetch them from a substituter). I don't think that e.g. using a local cache on a LAN to share your zfs.ko build between machines is any more problematic than creating it in the first place | 02:56:25 |
| emily | that's how it works in Nix/Nixpkgs; Hydra won't build things with non-free licences, so they are never substituted from the default substituters configuration, but nothing prevents you from caching them yourself | 02:57:10 |
| apteryx | you may still want CI to be able to build something to detect problems, but not distributing it as a substitute | 02:59:05 |
| apteryx | or in the case of the zfs combination with linux in an initrd, a server could have cached such initrd because it's using zfs itself for its file system, etc. | 02:59:43 |
| apteryx | so it seems to have value to be able to tag a package with 'this is for building locally only, in all situations' | 03:00:10 |
| emily | fair enough, if you have logic to avoid pushing !allowSubstitutes derivations to a cache etc. | 03:00:37 |
| emily | though I still think you only need it on the "push" end, not the "pull", where the issues of the mechanism arise :) | 03:00:48 |
| apteryx | at least with the way things are currently working with Guix (and I assume, Nix), anything in the store is available as a substitute, so there's no fine control on the push. | 03:01:48 |
| apteryx | if I'm not mistaken | 03:02:23 |
| emily | right, so I don't understand how allowSubstitutes helps… anyone who knows the hash could download the illegal binary from the cache, even if your client avoids doing so by default, which seems bad | 03:02:47 |
| apteryx | good point | 03:03:51 |
| apteryx | I'll review that part of the infra (how /gnu/store things end up being served as nars) on the substitute servers. Perhaps I can add some logic to prevent things marked as non-substitutable ending up as a .nar ready to be served. | 03:06:44 |
| apteryx | thanks for the ideas! | 03:08:01 |
| emily | I feel it's the wrong mechanism for this still, but ok :) | 03:15:09 |
| apteryx | For you the best option would be to not build such binary at all? Or something else? | 03:19:44 |
| apteryx | or perhaps we need a #:distributable? argument, orthogonal of substitutability | 03:20:31 |
| emily | any mechanism about not distributing a problematic derivation output is wholly orthogonal to whether the client can try substituting it, which there's no reason to let a derivation block as there are always legitimate use-cases (private LAN etc.) and which forbidding gets in the way of ("can't cache a closure properly because one of them is marked as !allowSubstitutes because it's meant to be trivial so build inputs get pulled in anyway") | 03:21:38 |
| emily | as in, what allowSubstitutes is meant to do doesn't help solve the problem in question in any way, and what it's meant to do has largely (on the Nix side at least) turned out to hurt more than it helps | 03:22:14 |
| emily | (I'd be hesitant to have CI build stuff that's considered legally problematic enough to not be distributable in the first place, but I guess Guix is strict enough about licensing that something like zfs.ko is probably the limit of the risk there.) | 03:23:28 |
 mra | In reply to @emilazy:matrix.org
as in, what allowSubstitutes is meant to do doesn't help solve the problem in question in any way, and what it's meant to do has largely (on the Nix side at least) turned out to hurt more than it helps one question: another use of allowSubstitutes = 0 on Guix is for HPC packages, specifically those which have CPU-specific optimisations, so that a client doesn't substitute a package which is optimised for a different CPU. How does Nix handle this case? | 06:54:40 |
| mra | This is somewhat orthogonal to the distributability concern, I agree, but this is currently one of the main applications of non-substitutability for us. | 06:55:50 |
 Eli Kogan-Wang | In reply to @morgan.arnold:matrix.org
one question: another use of allowSubstitutes = 0 on Guix is for HPC packages, specifically those which have CPU-specific optimisations, so that a client doesn't substitute a package which is optimised for a different CPU. How does Nix handle this case? Are you asking about https://wiki.nixos.org/wiki/Build_flags?
See https://github.com/NixOS/nixpkgs/pull/202526#issue-1461820752 | 07:21:01 |
| mra | Oh, interesting. It just has to be specifically requested. That makes sense. | 07:30:28 |
| emily | In reply to @morgan.arnold:matrix.org
one question: another use of allowSubstitutes = 0 on Guix is for HPC packages, specifically those which have CPU-specific optimisations, so that a client doesn't substitute a package which is optimised for a different CPU. How does Nix handle this case? I don't understand how this would ever arise. different flags would mean different derivation hashes so you'd never get an incorrect substitution, right? | 14:12:12 |
