To be frank, one of my hidden pleasure was that the module system is turing complete, and that people do not have to learn about functional programming to use it in NixOS.

On that I agree - recursive merging is just equivalent to functions in expressivity, but it's more natural to write configurations this way (also makes things more inspectable and easier to override if done ).

and Cue has a lot of shortcomings when it comes to property checking I believe

Ah right, I believe it's quite hard to encode custom validator beyond the builtin combinators. I think you can do something like write validation functions in Go and then glue things together at the scripting layer but it's suddenly not that simple anymore

yannham: fwiw, dhall has import-boundary caching of the reduced AST

Yeah, it's already something. But by incremental I really mean, in one file, you have a big configuration with recursive fields (might not be actually recursive, just a DAG, but fields that depend on other fields at least). You change enable to true to enable = false or greetings to "Hello, Plutus", and you want to recompute the least amount of stuff. This is on another level.

And it turns out it’s a pretty expensive operation in many practical use-cases, because partially applied functions tend to blow up a lot

From what I recall, I think this is a problem quite specific to dhall: basically their normal form representation which is cached isn't able to represent sharing, so you have to substitute everything and you end up with sometimes exponential blow up.

but it’s a hard problem, no?

Honestly, I don't really know. Like this, I would be tempted to say: hash-consing could be one way (especially given that Dhall has semantic hash). Another way could be to not necessarily cache normal forms, but allow things with remaining let-bindings. But Gabriella is a smart person, so I'm most definitely missing something. Could also be due to a conflicting design constraint elsewhere that requires cached values to be normal forms (or at least makes it harder if it's not the case). Arnaud probably knows better

but it’s a hard problem, no?

Honestly, I don't really know. Like this, I would be tempted to say: hash-consing could be one way (especially given that Dhall has semantic hashes). Another way could be to not necessarily cache normal forms, but allow things with remaining let-bindings. But Gabriella is a smart person, so I'm most definitely missing something. Could also be due to a conflicting design constraint elsewhere that requires cached values to be normal forms (or at least makes it harder if it's not the case). Arnaud probably knows better

In reply to @phaer:matrix.org
Yes, "just" the implementation work left 😄