!djTaTBQyWEPRQxrPTb:nixos.org

Nixpkgs Architecture Team

227 Members
https://github.com/nixpkgs-architecture, weekly public meetings on Wednesday 15:00-16:00 UTC at https://meet.jit.si/nixpkgs-architecture53 Servers

Load older messages


SenderMessageTime
12 Jul 2023
@nbp:mozilla.orgnbpThen I do not know, nor see how it would be different than today.11:05:14
@piegames:matrix.org@piegames:matrix.org Well the situation of today is pretty bad, so I hope it would be different 11:06:31
@hexa:lossy.networkhexasimple paths when13:33:53
@phaer:matrix.orgphaerRFC is merged πŸŽ‰13:46:11
@phaer:matrix.orgphaeras of 12 mins ago https://github.com/NixOS/rfcs/pull/140#event-979957369013:46:28
@piegames:matrix.org@piegames:matrix.orgYes.13:46:39
* @piegames:matrix.org@piegames:matrix.org is hovering over the green button at the implementation PR13:46:57
@phaer:matrix.orgphaerYes, "just" the implementation work left πŸ˜„13:47:50
@phaer:matrix.orgphaerhttps://github.com/NixOS/nixpkgs/pull/237439 13:48:08
@yannham:matrix.org@yannham:matrix.org

Cue suffers from the same slowness problems as the nixos module system afaik

On the performance matter, it's not really related to modules per se but in general I personally have high hopes in the incremental evaluation approach. It's less of an issue that your evaluation from scratch is slow, under the hypothesis that each new generation is just a small diff from the previous one, and that your interpreter can reuse most of the previous results incrementally. Lazy evaluation turns out to play quite well with incremental evaluation (basically, you can more or less swap the evaluator with a one performing various caching strategies without changing the semantics of the language). But hard to tell before trying this at scale.

The problem is unscoped merging, aka if you want to figure out whether something exists, you have to potentially search through every module definition

I might be misunderstanding, but I feel like it's more of a NixOS module system design decision (put everything in one big fixpoint soup), rather than something inherent to merging as defined e.g. in CUE ? Couldn't you build a more scoped/hierarchical system based on CUE for example (or even in pure Nix)?

14:59:02
@nbp:mozilla.orgnbp yannham: Actually, one case where maximal-laziness this would shine a lot is on the nixos/maintainers/option-usages.nix expression, where NixOS evaluation is re-done completely with one option changed, and evaluating whether one option is used as part of the computation. This was one of my assumption, until I realize that the feature got removed :( 15:25:18
@nbp:mozilla.orgnbpWhat is CUE?15:26:16
@yannham:matrix.org@yannham:matrix.orgCUE is a configuration language, non Turing-complete, with a very interesting approach (https://cuelang.org/). I would suspect it's not expressive enough for the Nix use-case, but they have a "meet" operator that is close to the fixpoint-merging done by the NixOS module system and Nickel's merging (and, for that matter, Jsonnet's inheritance '+' operator)15:28:54
@yannham:matrix.org@yannham:matrix.orgYeah, maximal laziness (plus a format to write the cached values on disk) would achieve that. I wonder if hash-consing and caching every little thing is the right tradeoff though, it might be quite costly and even a pessimization on big expressions. I suspect something slightly less fine-grained, such as caching at the level of attributes (recursively) and let bindings might work better. It's totally. It's totally speculative though, not backed by evidence15:37:28
@yannham:matrix.org@yannham:matrix.org * Yeah, maximal laziness (plus a format to write the cached values on disk) would achieve that. I wonder if hash-consing and caching every little thing is the right tradeoff though, it might be quite costly and even a pessimization on big expressions. I suspect something slightly less fine-grained, such as caching at the level of attributes (recursively) and let bindings might work better. It's totally speculative though, not backed by evidence15:37:46
@raitobezarius:matrix.orgraitobezarius
In reply to @yannham:matrix.org
CUE is a configuration language, non Turing-complete, with a very interesting approach (https://cuelang.org/). I would suspect it's not expressive enough for the Nix use-case, but they have a "meet" operator that is close to the fixpoint-merging done by the NixOS module system and Nickel's merging (and, for that matter, Jsonnet's inheritance '+' operator)
FWIW, I authored a "writeCueValidator" in nixpkgs
15:38:10
@raitobezarius:matrix.orgraitobezariuswe use for bootspec15:38:12
@nbp:mozilla.orgnbpTo be frank, one of my hidden pleasure was that the module system is turing complete, and that people do not have to learn about functional programming to use it in NixOS.15:38:14
@raitobezarius:matrix.orgraitobezarius it's optional behind bootspec.enableValidation because it pulls Go and this is not really acceptable for system build 15:38:31
@nbp:mozilla.orgnbp(except if you want to go beyond its interface and extend it, of course)15:38:45
@raitobezarius:matrix.orgraitobezariusand Cue has a lot of shortcomings when it comes to property checking I believe15:40:02
@raitobezarius:matrix.orgraitobezariussome stuff is hard to spec15:40:09
@yannham:matrix.org@yannham:matrix.org

To be frank, one of my hidden pleasure was that the module system is turing complete, and that people do not have to learn about functional programming to use it in NixOS.

On that I agree - recursive merging is just equivalent to functions in expressivity, but it's more natural to write configurations this way (also makes things more inspectable and easier to override if done ).

15:42:20
@yannham:matrix.org@yannham:matrix.org

and Cue has a lot of shortcomings when it comes to property checking I believe

Ah right, I believe it's quite hard to encode custom validator beyond the builtin combinators. I think you can do something like write validation functions in Go and then glue things together at the scripting layer but it's suddenly not that simple anymore

15:45:30
@profpatsch:augsburg.oneprofpatschyannham: fwiw, dhall has import-boundary caching of the reduced AST15:45:33
@profpatsch:augsburg.oneprofpatschAnd it turns out it’s a pretty expensive operation in many practical use-cases, because partially applied functions tend to blow up a lot15:46:00
@profpatsch:augsburg.oneprofpatschwhich is something the coq-people had figured out years ago from what I heard15:46:43
@nbp:mozilla.orgnbp yannham: Nix achieved maximal laziness in the past using the ATerm library (which apparently is no longer packaged in Nixpkgs πŸ˜‚). The principle of it was based on hash-cons.
One thought I had before, was that the Nix evaluator could hash the expression, the environment, and just cache the result of the top-level evaluations.
15:46:53
@profpatsch:augsburg.oneprofpatschAt least aspiwack said something along those lines15:47:03
@nbp:mozilla.orgnbpThis While this might not work as well on NixOS, this might save a lot on function applications for Nixpkgs.15:47:50

Show newer messages


Back to Room ListRoom Version: 9