Maybe the correct implementation is to define any "optional" options outside the freeform submodule, and then copy them in if defined?

# Set settings.foo to foo, if defined
optionalAttrs options.something.foo.isDefined {
  # Use mkDerivedConfig to also copy definition priority
  something.settings.foo = lib.mkDerivedConfig options.something.foo (v: v);
}

Perhaps submodule sub-options are only intended to be used when the value should always be in the generated value?

Maybe the correct implementation is to define any "optional" options outside the freeform submodule, and then copy them in if defined?

# Set settings.foo to foo, if defined
lib.optionalAttrs options.something.foo.isDefined {
  # Use mkDerivedConfig to also copy definition priority
  something.settings.foo = lib.mkDerivedConfig options.something.foo (v: v);
}

Perhaps submodule sub-options are only intended to be used when the value should always be in the generated value?

Thanks, those links are insightful!

I think 158598 is also related, but 158594 sounds like exactly what I want:

We could add a mkOption argument that has the effect of producing no default value and no config attribute when no definitions are present for the option.

But it looks like the latest comment is backtracking or downscaling that idea...

Playing around in the repl, I can get _module.freeformType and I can get any _freeformOptions for freeform submodules (freeformType = attrsOf submodule), but I can't seem to find the freeform value or definitions anywhere separate from the fully merged option value/defs. Can you confirm I'm not missing anything here?

I guess to do this I would have to write my own types.submodule equivalent (e.g. submoduleWithoutUndefs)? With a custom type I should be able to merge using evalModules but filter out any undefined options 🤔

Seems similar in principle to the types.record proposal in #257511, although I'd need optional and nested fields which I think is out of scope for that one.

Thanks, those links are insightful!

158594 sounds like exactly what I want:

We could add a mkOption argument that has the effect of producing no default value and no config attribute when no definitions are present for the option.

But it looks like the latest comment is backtracking or downscaling that idea...

Playing around in the repl, I can get _module.freeformType and I can get any _freeformOptions for freeform submodules (freeformType = attrsOf submodule), but I can't seem to find the freeform value or definitions anywhere separate from the fully merged option value/defs. Can you confirm I'm not missing anything here?

I guess to do this I would have to write my own types.submodule equivalent (e.g. submoduleWithoutUndefs)? With a custom type I should be able to merge using evalModules but filter out any undefined options 🤔

Seems similar in principle to the types.record proposal in #257511, although I'd need optional and nested fields which I think is out of scope for that one.

I guess to do this I would have to write my own types.submodule equivalent (e.g. submoduleWithoutUndefs)? With a custom type I should be able to merge using evalModules but filter out any undefined options 🤔

Or maybe this wouldn't work, if evalModules itself is what merges the option+freeform defs together

Seems it is: freeformConfig is evaluated here, but isn't exposed separately from config.

Would a PR be welcome to have evalModules include declaredConfig, freeformConfig and config separately in the evaluated result?

@mattsturg:matrix.org Immediate thoughts are that it might be a bit of a leaky abstraction, but maybe also not. Feel free to make a PR, but I can't promise to get to review it myself soon, but perhaps @roberthensing:matrix.org can :)

I've been playing around with this in this branch (current commit), but what I have now is infinitely recursive, because configs shape is changing based on options._module.definedOptionsOnly, but options itself depends on config.

Is this approach fundamentally flawed? Do you have any suggestions for working around the inf-recursion?

The immediate alternative that springs to mind is to pass both a config and a definedConfig to the modules and include definedConfig in the result, alongside config. This is more inline with my original proposal, but feels like a worse abstraction than having a _module.definedOptionsOnly option.

I definedOptionsOnly could also be an evalModules argument instead of an option, but that isn't really ideal either.

I've been playing around with this in this branch (current commit), but what I have now is infinitely recursive, because configs shape is changing based on options._module.definedOptionsOnly, but options itself recursively depends on config.

Is this approach fundamentally flawed? Do you have any suggestions for working around the inf-recursion?

The immediate alternative that springs to mind is to pass both a config and a definedConfig to the modules and include definedConfig in the result, alongside config. This is more inline with my original proposal, but feels like a worse abstraction than having a _module.definedOptionsOnly option.

I definedOptionsOnly could also be an evalModules argument instead of an option, but that isn't really ideal either.

I've been playing around with this in this branch (current commit), but what I have now is infinitely recursive, because configs shape is changing based on options._module.definedOptionsOnly, but options itself recursively depends on config.

Is this approach fundamentally flawed? Do you have any suggestions for working around the inf-recursion?

The immediate alternative that springs to mind is to pass both a config and a definedConfig to the modules and include definedConfig in the result, alongside config. This is more inline with my original proposal, but feels like a worse abstraction than having a _module.definedOptionsOnly option.

definedOptionsOnly could also be an evalModules argument instead of an option, but that isn't really ideal either.

infinisil Robert Hensing (roberth) Thanks for your support so far!

I've been playing around with the concept in this branch (current commit), but my current implementation is infinitely recursive, because configs shape is changing based on options._module.definedOptionsOnly, but options itself recursively depends on config.

Is this approach fundamentally flawed? Do you have any suggestions for working around the inf-recursion?

The immediate alternative that springs to mind is to pass both a config and a definedConfig to the modules and include definedConfig in the result, alongside config. This is more inline with my original proposal, but feels like a worse abstraction than having a _module.definedOptionsOnly option.

definedOptionsOnly could also be an evalModules argument instead of an option, but that isn't really ideal either.

I've opened a draft PR that does it as an evalModules arg, so it actually works without inf-recursion. I detailed this limitation in the PR description. Any feedback would be greatly appreciated!

https://github.com/NixOS/nixpkgs/pull/333799

I also don't know how much time I can dedicate to it. I had looked at it before when exploring solutions to https://matrix.to/#/!wfudwzqQUiJYJnqfSY:nixos.org/$mVuETvj4dPJNF6ZUdVEVvWmp07L5G3X-6y0bk-NmCP4?via=nixos.org&via=matrix.org&via=nixos.dev, however I came to the conclusion it is an interesting (and useful looking) type, but not a viable solution to the problem we have in nixvim.

I'll attempt to clarify a little in my response to your (very helpful) feedback in #333799.

Thanks again Robert Hensing (roberth) for your feedback and for reminding me of your existing proposals!

I've posted my initial thoughts on github. And further;

Reading through your simple/minimal module eval proposal, I think that's a great solution provided defining config from outside the simple-module type looks the same as defining config for a submodule type; i.e. many of the features you mention arent supported (mkIf for instance) still work, but are handled in the "host" module system before evaluating the resulting configured modules.

Enforcing one module/option per attr level (i.e. no nested options) sounds like a reasonable simplification too.

My main concern is that "perfection" here may be acting as the enemy of "good"; i.e. your proposal is great, but will be much more effort to actually get merged into nixpkgs.

From a different perspective, there's also some simplicity in not reinventing the wheel here, just to support one small feature (config without undefined opts). While we could see this as an opportunity to design a better/faster system, we could also just use the small change for now and consider in the future how it can be improved/replaced. 🤔

Hm, I'm not sure why I was under the impression that the record type was restricted to being flat (and can't contain sub-options or sub-records, I guess). I assume the wildcard type could also be set to something like attrsOf anything to allow recursive merging?

If we can define nested optionalFields and recursively merged wildcards I think it fits nixvim's needs.

Hm, I'm not sure why I was under the impression that the record type was restricted to being flat (and couldn't contain sub-options, sub-records, sub-fields?). I assume the wildcard type could also be set to something like attrsOf anything to allow recursive merging?

If we can define nested optionalFields and recursively merged wildcards I think it fits nixvim's needs.