Nixpkgs Architecture Team | 227 Members | |
| https://github.com/nixpkgs-architecture, weekly public meetings on Wednesday 15:00-16:00 UTC at https://meet.jit.si/nixpkgs-architecture | 53 Servers |
| Sender | Message | Time |
|---|---|---|
| 12 Jul 2023 | ||
| Then I do not know, nor see how it would be different than today. | 11:05:14 | |
| Well the situation of today is pretty bad, so I hope it would be different | 11:06:31 | |
| simple paths when | 13:33:53 | |
| RFC is merged π | 13:46:11 | |
| as of 12 mins ago https://github.com/NixOS/rfcs/pull/140#event-9799573690 | 13:46:28 | |
| Yes. | 13:46:39 | |
| * | 13:46:57 | |
| Yes, "just" the implementation work left π | 13:47:50 | |
| https://github.com/NixOS/nixpkgs/pull/237439 | 13:48:08 | |
On the performance matter, it's not really related to modules per se but in general I personally have high hopes in the incremental evaluation approach. It's less of an issue that your evaluation from scratch is slow, under the hypothesis that each new generation is just a small diff from the previous one, and that your interpreter can reuse most of the previous results incrementally. Lazy evaluation turns out to play quite well with incremental evaluation (basically, you can more or less swap the evaluator with a one performing various caching strategies without changing the semantics of the language). But hard to tell before trying this at scale.
I might be misunderstanding, but I feel like it's more of a NixOS module system design decision (put everything in one big fixpoint soup), rather than something inherent to merging as defined e.g. in CUE ? Couldn't you build a more scoped/hierarchical system based on CUE for example (or even in pure Nix)? | 14:59:02 | |
| yannham: Actually, one case where maximal-laziness this would shine a lot is on the nixos/maintainers/option-usages.nix expression, where NixOS evaluation is re-done completely with one option changed, and evaluating whether one option is used as part of the computation. This was one of my assumption, until I realize that the feature got removed :( | 15:25:18 | |
| What is CUE? | 15:26:16 | |
| CUE is a configuration language, non Turing-complete, with a very interesting approach (https://cuelang.org/). I would suspect it's not expressive enough for the Nix use-case, but they have a "meet" operator that is close to the fixpoint-merging done by the NixOS module system and Nickel's merging (and, for that matter, Jsonnet's inheritance '+' operator) | 15:28:54 | |
| Yeah, maximal laziness (plus a format to write the cached values on disk) would achieve that. I wonder if hash-consing and caching every little thing is the right tradeoff though, it might be quite costly and even a pessimization on big expressions. I suspect something slightly less fine-grained, such as caching at the level of attributes (recursively) and let bindings might work better. It's totally. It's totally speculative though, not backed by evidence | 15:37:28 | |
| * Yeah, maximal laziness (plus a format to write the cached values on disk) would achieve that. I wonder if hash-consing and caching every little thing is the right tradeoff though, it might be quite costly and even a pessimization on big expressions. I suspect something slightly less fine-grained, such as caching at the level of attributes (recursively) and let bindings might work better. It's totally speculative though, not backed by evidence | 15:37:46 | |
In reply to @yannham:matrix.orgFWIW, I authored a "writeCueValidator" in nixpkgs | 15:38:10 | |
| we use for bootspec | 15:38:12 | |
| To be frank, one of my hidden pleasure was that the module system is turing complete, and that people do not have to learn about functional programming to use it in NixOS. | 15:38:14 | |
it's optional behind bootspec.enableValidation because it pulls Go and this is not really acceptable for system build | 15:38:31 | |
| (except if you want to go beyond its interface and extend it, of course) | 15:38:45 | |
| and Cue has a lot of shortcomings when it comes to property checking I believe | 15:40:02 | |
| some stuff is hard to spec | 15:40:09 | |
On that I agree - recursive merging is just equivalent to functions in expressivity, but it's more natural to write configurations this way (also makes things more inspectable and easier to override if done ). | 15:42:20 | |
Ah right, I believe it's quite hard to encode custom validator beyond the builtin combinators. I think you can do something like write validation functions in Go and then glue things together at the scripting layer but it's suddenly not that simple anymore | 15:45:30 | |
| yannham: fwiw, dhall has import-boundary caching of the reduced AST | 15:45:33 | |
| And it turns out itβs a pretty expensive operation in many practical use-cases, because partially applied functions tend to blow up a lot | 15:46:00 | |
| which is something the coq-people had figured out years ago from what I heard | 15:46:43 | |
| yannham: Nix achieved maximal laziness in the past using the ATerm library (which apparently is no longer packaged in Nixpkgs π). The principle of it was based on hash-cons. One thought I had before, was that the Nix evaluator could hash the expression, the environment, and just cache the result of the top-level evaluations. | 15:46:53 | |
| At least aspiwack said something along those lines | 15:47:03 | |
| This While this might not work as well on NixOS, this might save a lot on function applications for Nixpkgs. | 15:47:50 | |