!UKDpaKNNsBpOPfLWfX:zhaofeng.li

Colmena

329 Members
A simple, stateless NixOS deployment tool - https://github.com/zhaofengli/colmena115 Servers

Load older messages


SenderMessageTime
2 Sep 2022
@blaggacao:matrix.orgDavid Arnold (blaggacao)(which may be the main current justification)18:34:24
@linus:schreibt.jetztLinux Hackerman
In reply to @zhaofeng:zhaofeng.li

To further expand on that, consuming the existing nixosConfigurations is problematic, since Colmena needs to insert two things into the configuration: Modules (deployment.* options and configs) as well as module arguments (name and nodes which allows you to refer to configurations of other nodes in the cluster). This is in part following the design decisions of NixOps and morph. Inserting them cannot be cleanly done in case of nixosConfigurations which contains complete, evaluated configurations rather than individual NixOS modules.

We could remove this need and make the user specify the deployment.* configs externally, outside the NixOS configuration. This is the approach that deploy-rs takes (the deploy output). The downside is two-fold:

  1. The level of configuration composition possible will be limited. As an example, I set deployment.tags = [ "routers" ]; in my router module so enabling it will automatically make colmena apply --on @routers deploy to this node. Additionally, I use the added nodes argument extensively in my configuration to set up a WireGuard mesh and iBGP peerings between my nodes.
  2. The effect Colmena can have on the target node configuration will be limited. For instance, Colmena creates the ${name}-key.service for each secret file defined in deployment.keys, which will no longer be possible. deploy-rs uses a deploy binary copied alongside the profile to implement activation and other features.

The end result is a level of functionality noticeably different from "normal" Colmena that would necessitate a lot of documentation changes and still be confusing to users.

  1. The nodes argument doesn't require colmena though. nixosConfigurations can reference self.outputs.nixosConfigurations to get at other hosts.
  2. (covers deployment.tags as well) I'd prefer for there to be e.g. inputs.colmena.nixosModules.default that doesn't just supply the options (as inputs.colmena.nixosModules.deploymentOptions does), but also their implementation. I don't see (feel free to point it out) what needs to be injected there that isn't available at "any" evaluation time -- with nixops's model of resources that are also managed by the tool, that makes sense, but given that colmena doesn't "create" machines or otherwise manage resources (sensible choice IMO!) it's not necessary.
18:35:45
@linus:schreibt.jetztLinux Hackerman
In reply to @zhaofeng:zhaofeng.li

To further expand on that, consuming the existing nixosConfigurations is problematic, since Colmena needs to insert two things into the configuration: Modules (deployment.* options and configs) as well as module arguments (name and nodes which allows you to refer to configurations of other nodes in the cluster). This is in part following the design decisions of NixOps and morph. Inserting them cannot be cleanly done in case of nixosConfigurations which contains complete, evaluated configurations rather than individual NixOS modules.

We could remove this need and make the user specify the deployment.* configs externally, outside the NixOS configuration. This is the approach that deploy-rs takes (the deploy output). The downside is two-fold:

  1. The level of configuration composition possible will be limited. As an example, I set deployment.tags = [ "routers" ]; in my router module so enabling it will automatically make colmena apply --on @routers deploy to this node. Additionally, I use the added nodes argument extensively in my configuration to set up a WireGuard mesh and iBGP peerings between my nodes.
  2. The effect Colmena can have on the target node configuration will be limited. For instance, Colmena creates the ${name}-key.service for each secret file defined in deployment.keys, which will no longer be possible. deploy-rs uses a deploy binary copied alongside the profile to implement activation and other features.

The end result is a level of functionality noticeably different from "normal" Colmena that would necessitate a lot of documentation changes and still be confusing to users.

*
  1. The nodes argument doesn't require colmena though. nixosConfigurations can reference self.outputs.nixosConfigurations to get at other hosts.
  2. (covers deployment.tags as well) I'd prefer for there to be e.g. inputs.colmena.nixosModules.default that doesn't just supply the options (as inputs.colmena.nixosModules.deploymentOptions does), but also their implementation. I don't see (feel free to point it out if I'm missing something) what needs to be injected there that isn't available at "any" evaluation time -- with nixops's model of resources that are also managed by the tool, that makes sense, but given that colmena doesn't "create" machines or otherwise manage resources (sensible choice IMO!) it's not necessary.
18:35:53
@blaggacao:matrix.orgDavid Arnold (blaggacao)(I might be totally wrong, too)18:36:10
@yuka:yuka.dev@yuka:yuka.dev
In reply to @blaggacao:matrix.org
But, I don't know if that may break the paraleval
I don't think that would be influenced by this kind of change
18:38:13
@linus:schreibt.jetztLinux Hackermansame18:38:38
@yuka:yuka.dev@yuka:yuka.devI will try to prepare a PR that solves most of the problem without too many changes18:38:40
@yuka:yuka.dev@yuka:yuka.devIf that's fine with y'all18:38:51
@yuka:yuka.dev@yuka:yuka.devfirst not even focusing on producing a nixosConfigurations, but giving a first stepping stone in that direction18:39:14
@zhaofeng:zhaofeng.liZhaofeng Li
In reply to @yuka:yuka.dev
You could just tell the users that the colmena binary has to be from the exact same version as the hive expression
There still need to be some kind of versioning and/or a contract, because David Arnold (blaggacao) is making a framework (std) that will emit this structure with its own logic
18:41:35
@yuka:yuka.dev@yuka:yuka.devI mean, how is the current interface versioned?18:42:16
@zhaofeng:zhaofeng.liZhaofeng LiThe current interface isn't versioned at all and the expressions are embedded into the Colmena executable18:43:44
@yuka:yuka.dev@yuka:yuka.devI mean the interface between that expression and my flake18:44:03
@yuka:yuka.dev@yuka:yuka.devThere is still an interface, even if it's at a different level18:44:11
@linus:schreibt.jetztLinux Hackerman (aside: an interface that I'd love to see changed, to avoid mixing the nodes with special attrs like defaults and meta!) 18:45:07
@zhaofeng:zhaofeng.liZhaofeng Li
In reply to @yuka:yuka.dev
I mean the interface between that expression and my flake
That's also poorly defined at the moment, documented mostly as examples. One possibility is to use the NixOS module system to define the schema, which is what NixOps may be heading towards: https://github.com/NixOS/nixops/pull/1508
18:47:31
@linus:schreibt.jetztLinux Hackermannixos modules for that would be great, I think that approach was pioneered by infinisil in nixus18:48:21
@linus:schreibt.jetztLinux Hackermandefining modules that have the entire deployment as their scope instead of an individual host is 😘🤌18:49:15
@yuka:yuka.dev@yuka:yuka.devdefinitely, but I'd like to not mix up multiple issues/improvements too much18:49:49
@linus:schreibt.jetztLinux Hackermanyeah. I'm also not sure that should be colmena's responsibility.18:50:11
@yuka:yuka.dev@yuka:yuka.devhttps://github.com/yu-re-ka/colmena/commit/cd558fee7f4251719d1101f9677dce2581f6f58e19:19:20
@yuka:yuka.dev@yuka:yuka.devThis is enough changes to make it work for me19:19:34
@yuka:yuka.dev@yuka:yuka.dev

In my flake, I now do:

  outputs = { self, nixpkgs, colmena, ... }@inputs: {
    colmenaEval = colmena.evalHive {
      # whatever was in outputs.colmena before
    };
    nixosConfigurations = self.outputs.colmenaEval.nodes;
  };
20:11:45
@yuka:yuka.dev@yuka:yuka.devthough somehow it does not like to do pure eval20:12:14
@zhaofeng:zhaofeng.liZhaofeng Li Yeah, if the git repo is dirty, nix flake metadata will not return a locked flake URI that's usable in pure evaluation mode. That's one gotcha I encountered when looking into pure eval. 20:13:32
@yuka:yuka.dev@yuka:yuka.devDoes the assets wrapper flake solve this?20:14:10
@zhaofeng:zhaofeng.liZhaofeng LiYes, the locked URI of the assets flake encompasses the inputs as well.20:16:01
@zhaofeng:zhaofeng.liZhaofeng LiYour solution should work if the git repo is clean20:16:35
@yuka:yuka.dev@yuka:yuka.dev
In reply to @zhaofeng:zhaofeng.li
Yes, the locked URI of the assets flake encompasses the inputs as well.
also if it is a dirty git repo
20:16:50
@yuka:yuka.dev@yuka:yuka.dev
In reply to @zhaofeng:zhaofeng.li
Yes, the locked URI of the assets flake encompasses the inputs as well.
* also if one of the inputs is a dirty git repo?
20:16:56

Show newer messages


Back to Room ListRoom Version: 6