| 24 Aug 2021 |
 andi- | Running nix-build as a regular user right now does eval + drives the build in a single process, correct? I was thinking of doing the eval in a forked child such that we can (safely?) discard all the memory from the eval that we don't have to hold on to anymore. Are there any obvious roadblocks that I'd run into? | 15:13:51 |
 pamplemousse |
does eval + drives the build in a single process, correct?
AFAICT, yes; And it uses threads for parallelism.
| 18:51:57 |
| pamplemousse | andi-: FYI, I considered doing something similar to be able to fuzz the parsing / evaluation part of nix (because it's not reentrant - i.e. mutates a global state)
I don't remember exactly why I abandoned this, but speed was definitely a problem (fork/exec is much slower - but it might not be problematic in your use case). | 18:55:03 |
| pamplemousse | (it kinda defeated the purposed of having an "in-memory" fuzzer) | 18:55:31 |
| andi- | Yeah for fuzzing you probably want something much faster even thought you could fuzz multiple cases in one process (as long as you keep the history of invocations to minimize the case afterwards) | 18:56:05 |
| pamplemousse | In reply to @andi:kack.it
Yeah for fuzzing you probably want something much faster even thought you could fuzz multiple cases in one process (as long as you keep the history of invocations to minimize the case afterwards) The problem is that it raises a lot of false positives this way in practice; Every "next" run is corrupted by dangling global state left by the previous... | 19:02:40 |
| pamplemousse | (I am really talking about nix specifically here) | 19:03:01 |
| andi- | Yeah, I'd only run like 3-5 or so per fork. That is still a 5x speedup? | 21:08:42 |
| andi- | Another thing that we talked about earlier in #infrastructure:nixos.org : Could we tell the kernel / emulate the load in the sandbox so it only covers the load produced from the contained jobs? That topic came up as running many builds with a small number of cores might always be limited by load if the build system is load aware (e.g. ninja & gnumake with -l). lxc apparently does something like that. The open question was if the kernel is able to do that or else if we could/should do that in userspace (in the nix-daemon). | 22:11:22 |
| 25 Aug 2021 |
 Vladimír Čunát | I'm actually not sure it's a good idea. If the CPU is shared, strictly partitioning it doesn't sound like a good approach for overall efficiency... though it depends how exactly we'd use it. | 18:55:00 |
|  trofi joined the room. | 21:42:44 |
| trofi set a profile picture. | 22:19:15 |
| 26 Aug 2021 |
| trofi changed their display name from Sergei Trofimovich to trofi. | 07:41:05 |
 Las | Does anyone know why Recursive Nix is experimental? | 08:58:54 |
 Gytis Ivaskevicius | prob due to RFC 92 | 17:06:45 |
| Gytis Ivaskevicius | it implements similar feature but it all is approached differently | 17:07:04 |
 tomberek | (i could be wrong here) I think there is an issue about being in a build, and then starting another one which blocks the original build until the internal one completes. | 17:07:56 |
| tomberek | If you have overuse of recursive nix, this means you will have long chains of blocked builds. This is compared to the normal situation where a dependency can complete, tear down the sandbox, store results into cache, etc, and then start the next build whenever you want. | 17:09:29 |
| tomberek | My mental model for this is similar to "tail-recursion optimization". That if you know that the recursion is a tail call, you can do avoid blowing up the stack by turning a function call into a jump. Recursive-nix allows for arbitrary recursion, RFC92 tries to enforce the "tail-call" style. | 17:11:56 |
| tomberek | * My mental model for this is similar to "tail call optimization". That if you know that the recursion is a tail call, you can do avoid blowing up the stack by turning a function call into a jump. Recursive-nix allows for arbitrary recursion, RFC92 tries to enforce the "tail-call" style. | 17:12:23 |
| Gytis Ivaskevicius | Not sure if thats how i'd call it. I'd say its more of a build scheduling | 17:13:51 |
| Gytis Ivaskevicius | may need to read it again | 17:13:57 |
 sterni | In reply to @gytis-ivaskevicius:matrix.org
prob due to RFC 92 RFC 92 depends on recursive nix (albeit a more limited variant of it) | 17:48:20 |
| Gytis Ivaskevicius | In reply to @sternenseemann:systemli.org
RFC 92 depends on recursive nix (albeit a more limited variant of it) yeah, John Ericson idea is "STOP WRITING SHIT CODE!!!!" :D | 17:51:32 |
| Gytis Ivaskevicius | (sorry for the ping) | 17:51:42 |
| Gytis Ivaskevicius | (Actually I'd love to hear some update on all that 😉 ) | 17:52:32 |
 John Ericson | :) | 17:52:47 |
| John Ericson | Only news is we need more people to sign up to be shepherds | 17:54:22 |
| Gytis Ivaskevicius | I'm afraid its too big brain for me | 17:55:10 |
| tomberek | btw: i took a brief stab at updating the eelco commit into the new approach of DerivationTypes, but i started wading in areas of code I think John would be 10x faster than me on. | 17:55:25 |
