nbp: thanks for the feedback! First I want to say I'm convinced these contracts will really help in nixpkgs but I'm still experimenting on the UX side. So don't take my mkOption experience as something set in stone. I made other contracts without those and they work better I find, so I'll probably remove that experiment at some point.

I considered enums at the start, but the problem I have with those is it's not easily extendable by the end user without modifying nixpkgs directly. At least I don't think so?

What I have currently is the following. My "best" and most recent contracts are currently the backup and database backup contracts. The former is for backing up files, the latter for backing up the output of a command (like pg_dump) without going through an intermediate file. (writing this down, I should probably find a better name than database backup)

This is the backup contract and the database backup one I have now. They're both just attrsets with some pre-defined schema: { request = submodule { ... }; result = submodule { ... }; }. (I use requestType and resultType for the database backup contract, but I like it less).
The backup contract essentially takes in a list of directories (in the request attribute) and produces (in the result attribute) a backup systemd service and a restore script. This is quite generic and describes IMO the essence of backing up something. For the database backup, it takes in a backup command and a restore command and again produces the backup service and the restore script.

A requester of the backup contract, for example Nextcloud which wants to be backed up, would then provide a shb.nextcloud.backup attribute which uses the requestType. The trick here is that the Nextcloud module fills out the default values of the type. You can see it sets the attributes expected by the contract - user, sourceDirectories and excludePatterns - to something that suits Nextcloud.

A provider of the backup contract, for example Restic, would provide an attribute that uses the contracts.backup.request type and the contracts.backup.result type, here the shb.restic.instances attribute. Here again I use the default values trick to let Restic set the correct name for the backup script and restore script in the result attribute. You can also see that Restic added a settings attribute next to the request and result ones from the contract. This is where each provider can add specific options that are essential for the provider (like for Restic, you must configure the repository where to backup) but are not mandated by the contract.
It would be cool to actually have an enforced schema with this request, result and settings trio but I didn't think yet if it's a good idea nor how it would work.

Putting both sides of the contract together is the end user's responsibility. Here, for backing up Nextcloud files with Restic, it looks like so:

shb.restic.instances."nextcloud" = {
  request = config.shb.nextcloud.backup;
  settings.enable = true;
};

Assuming settings.enable defaults to true, you get a one-liner:

shb.restic.instances."nextcloud".request = config.shb.nextcloud.backup;

The beauty here is that if you had a Vaultwarden instance you want to backup, you can just add the following snippet without knowing anything about how to backup Vaultwarden:

shb.restic.instances."vaultwarden".request = config.shb.vaultwarden.backup;

Similarly, if you wanted to also use BorgBackup to backup Nextcloud and Vaultwarden, you can also just copy paste the snippets and just s/restic/borgbackup:

shb.restic.instances."nextcloud".request = config.shb.nextcloud.backup;
shb.restic.instances."vaultwarden".request = config.shb.vaultwarden.backup;
shb.borgbackup.instances."nextcloud".request = config.shb.nextcloud.backup;
shb.borgbackup.instances."vaultwarden".request = config.shb.vaultwarden.backup;

Of course, you'd need to configure Nextcloud, Vaultwarden, Restic and BorgBackup with specific options. But this snippet is just showing the pluming going on.

If you're curious, backing up Vaultwarden is not necessarily complicated. But still, hiding it from the end user is nice.

Using the database backup contract is as easy:

shb.borgbackup.databases."postgres".request = config.shb.postgresql.databasebackup;

Here's the PostgreSQL provider side if you're curious.

But now comes the even better part IMO. How do you make sure the Restic and BorgBackup are equivalent (from the contract perspective)? With generic tests! Here's the one for backup contract and here's the one for database backup contract. They are generic because they're a function taking in a location to a module so you plug in a module that provides the contract and the test will make sure it has the same behavior. Of course you actually need to call that function to test something and that is done here and here respectively for the backup and database backup contract.

These generic tests have a lot of value IMO.

So yeah, that's where I'm at with contracts. If you spelunk in the code of the project, you'll see I have a contract for secrets and ssl. They predate this request, result and settings trio so I'll update them at some point.

One last thing, you'll note that the restore script and backup service in the result part of either backup contract is not really used. That is true. I mostly needed this to make the generic test actually work. But they're still nice for documentation if anything. One result that is actually used is for the secrets contract. Everything I explained above stays the same for this contract, if you want to see how the result is used (which for the secrets contract is the location of the secret on the filesystem), you can look at this doc I wrote.

nbp: thanks for the feedback! First I want to say I'm convinced these contracts will really help in nixpkgs but I'm still experimenting on the UX side. So don't take my mkOption experience as something set in stone. I made other contracts without those and they work better I find, so I'll probably remove that experiment at some point.

I considered enums at the start, but the problem I have with those is it's not easily extendable by the end user without modifying nixpkgs directly. At least I don't think so?

What I have currently is the following. My "best" and most recent contracts are currently the backup and database backup contracts. The former is for backing up files, the latter for backing up the output of a command (like pg_dump) without going through an intermediate file. (writing this down, I should probably find a better name than database backup)

This is the backup contract and the database backup one I have now. They're both just attrsets with some pre-defined schema: { request = submodule { ... }; result = submodule { ... }; }. (I use requestType and resultType for the database backup contract, but I like it less).
The backup contract essentially takes in a list of directories (in the request attribute) and produces (in the result attribute) a backup systemd service and a restore script. This is quite generic and describes IMO the essence of backing up something. For the database backup, it takes in a backup command and a restore command and again produces the backup service and the restore script.

A requester of the backup contract, for example Nextcloud which wants to be backed up, would then provide a shb.nextcloud.backup attribute which uses the requestType. The trick here is that the Nextcloud module fills out the default values of the type. You can see it sets the attributes expected by the contract - user, sourceDirectories and excludePatterns - to something that suits Nextcloud.

A provider of the backup contract, for example Restic, would provide an attribute that uses the contracts.backup.request type and the contracts.backup.result type, here the shb.restic.instances attribute. Here again I use the default values trick to let Restic set the correct name for the backup script and restore script in the result attribute. You can also see that Restic added a settings attribute next to the request and result ones from the contract. This is where each provider can add specific options that are essential for the provider (like for Restic, you must configure the repository where to backup) but are not mandated by the contract.
It would be cool to actually have an enforced schema with this request, result and settings trio but I didn't think yet if it's a good idea nor how it would work.

Putting both sides of the contract together is the end user's responsibility. Here, for backing up Nextcloud files with Restic, it looks like so:

shb.restic.instances."nextcloud" = {
  request = config.shb.nextcloud.backup;
  settings.enable = true;
};

Assuming settings.enable defaults to true, you get a one-liner:

shb.restic.instances."nextcloud".request = config.shb.nextcloud.backup;

The beauty here is that if you had a Vaultwarden instance you want to backup, you can just add the following snippet without knowing anything about how to backup Vaultwarden:

shb.restic.instances."vaultwarden".request = config.shb.vaultwarden.backup;

Similarly, if you wanted to also use BorgBackup to backup Nextcloud and Vaultwarden, you can also just copy paste the snippets and just s/restic/borgbackup:

shb.restic.instances."nextcloud".request = config.shb.nextcloud.backup;
shb.restic.instances."vaultwarden".request = config.shb.vaultwarden.backup;
shb.borgbackup.instances."nextcloud".request = config.shb.nextcloud.backup;
shb.borgbackup.instances."vaultwarden".request = config.shb.vaultwarden.backup;

Of course, you'd need to configure Nextcloud, Vaultwarden, Restic and BorgBackup with specific options. But this snippet is just showing the pluming going on.

If you're curious, backing up Vaultwarden is not necessarily complicated. But still, hiding it from the end user is nice.

Using the database backup contract is as easy:

shb.borgbackup.databases."postgres".request = config.shb.postgresql.databasebackup;

Here's the PostgreSQL provider side if you're curious.

But now comes the even better part IMO. How do you make sure the Restic and BorgBackup are equivalent (from the contract perspective)? With generic tests! Here's the one for backup contract and here's the one for database backup contract. They are generic because they're a function taking in a location to a module so you plug in a module that provides the contract and the test will make sure it has the same behavior. Of course you actually need to call that function to test something and that is done here for the backup and database backup contract.

These generic tests have a lot of value IMO.

So yeah, that's where I'm at with contracts. If you spelunk in the code of the project, you'll see I have a contract for secrets and ssl. They predate this request, result and settings trio so I'll update them at some point.

One last thing, you'll note that the restore script and backup service in the result part of either backup contract is not really used. That is true. I mostly needed this to make the generic test actually work. But they're still nice for documentation if anything. One result that is actually used is for the secrets contract. Everything I explained above stays the same for this contract, if you want to see how the result is used (which for the secrets contract is the location of the secret on the filesystem), you can look at this doc I wrote.

enum are externally extendable in the module system. This is how desktopManagers / windowManagers are handled. Or why they got added a while back.
The idea of enums is that every module can add the new entry to enable it-self.

type = lib.types.enum [ "this-service-name" ];

makes sense, as opposed to many other languages, because options can be extended by multiple modules. Thus if all providers were to add extra declarations, you could have an option which enumerate all services which could be used as a provider.

Here is an example, where a module declare an enum with no "implementation":
https://github.com/NixOS/nixpkgs/blob/9956df1047bba4e000354f77229b0066d7f364e0/nixos/modules/virtualisation/podman/network-socket.nix#L33-L39

And another module provide an implementation for it:
https://github.com/NixOS/nixpkgs/blob/9956df1047bba4e000354f77229b0066d7f364e0/nixos/modules/virtualisation/podman/network-socket-ghostunnel.nix#L12C50-L18

Could I get some insight into how the module system evaluates a type attrs of differing priorities? I've been working on a little thing that has descending attrsets with reducing priority, in hopes that when they merge some lower attrsets would be erased due to the priority difference but that doesnt seem to be happening. Not sure if I'm operating on a misunderstanding of priorities and merging or not though.

Here's a very minimal example of my issue including a description of what I'm expecting to happen.

let
  lib = import <nixpkgs/lib>;
in lib.evalModules {
  modules = [( { lib, ... }: {
    options = {
      foo = lib.mkOption {
        type = with lib.types; attrs;
      };
    };

    config = lib.mkMerge [
      { foo.bar = lib.mkForce {}; }
      { foo.bar.baz = lib.mkForce {}; }
    ];
  })];
}

You can save the above and use nix eval --file name.nix config or nix repl to view what happens as well, but this is what I get on my system:

nix-repl> :p (import ./test.nix).config
{
  foo = {
    bar = {
      baz = {
        _type = "override";
        content = { };
        priority = 50;
      };
    };
  };
}

From what I understand about both priorities and merging, bar from foo.bar should have a priority of "50" as defined by lib.mkForce, whereas bar from foo.bar.baz should have whatever the default priority is, since foo.bar.baz is the same as foo = { bar = { baz = ...; }; };, which won't have any priority. In that case, I expected foo.bar's priority to overrule foo.bar.baz from merging baz into it, and yet for some reason it does.

The weirdest part is yet to come.. when I switch them around, as in declare foo.bar.baz first and then foo.bar next, it correctly overrides. What's going on here? As far as I know merging is commutative and overriding wouldn't have been affected by it, and yet I'm seeing a sign of it not being the case. This is why I'm first suspecting that I have a misunderstanding regarding how merging works with and without overrides. Does anyone have insight into why this happens?

Oh … how did I missed it …

Properties are evaluated at options, not after, which is why using the value { bar = lib.mkForce {}; } and the value { bar.baz = lib.mkForce {}; } appears in the result, and that we can have one override the other as the update operator defines. Simply because properties are not evaluated, but the value are simply merged.

nix-repl> opt = { options.foo = lib.mkOption { type = lib.types.attrs; }; }
nix-repl> d0 = { foo = lib.mkForce { bar = {}; }; }
nix-repl> d1 = { foo = lib.mkForce { bar.baz = {}; }; }
nix-repl> :p (lib.evalModules { modules = [ opt d0 d1 ]; }).config
{ foo = { bar = { }; }; }
nix-repl> :p (lib.evalModules { modules = [ opt d1 d0 ]; }).config
{ foo = { bar = { baz = { }; }; }; }

Which does not have the priority leaking in the value of the option.

In reply to @frontear:matrix.org
thanks for the response thats honestly somewhat embarrassing that i didnt catch that earlier

Hi all

I've come across a fun problem recently. I maintain a set of modules, and previously I had an option in a submodule, but now I've made it a regular (top-level) option. Per usual with changes like this, I wanted to use mkRenamedOptionModule to alias that option submodule option into the top-level option, but I got a nasty surprise with a cannot find attribute <old attribute name> error message

After doing some digging, I'm assuming this is because submodules sort of strip out the regular option data of their inner values, and only expose the fully evaluated configuration. Am I right here? Is there a good way around this?

Here's a minimum repro:

# repro.nix
let
  lib = import <nixpkgs/lib>;
in

lib.evalModules {
  modules = [
    {
      options = {
        # Create a submodule option
        aSubmodule = lib.mkOption {
          type = lib.types.submodule {
            options = {
              # With an inner option
              someSubmoduleOption = lib.mkEnableOption "something";
            };
          };

          default = { };
        };

        # And a top-level option
        aRegularOption = lib.mkEnableOption "something";
      };
    }

    # Alias them
    (lib.mkRenamedOptionModule [ "aSubmodule" "someSubmoduleOption" ] [ "aRegularOption" ])

    {
      # Test the alias
      # `nix eval --file repro.nix config.aRegularOption`
      aSubmodule.someSubmoduleOption = true;
      # ...but instead you get
      # error: evaluation aborted with the following error message: 'cannot find attribute `aSubmodule.someSubmoduleOption''
    }
  ];
}