!UKDpaKNNsBpOPfLWfX:zhaofeng.li

Colmena

323 Members
A simple, stateless NixOS deployment tool - https://github.com/zhaofengli/colmena110 Servers

Load older messages


SenderMessageTime
21 Oct 2022
@yuka:yuka.dev@yuka:yuka.dev
[ERROR] Failed to evaluate net-vm, monitor, tvpi, and 11 other nodes - Last 5 lines of logs:
[ERROR]  created)
[ERROR]    state) Running
[ERROR]   stderr) error: unrecognised flag '--expr'
[ERROR]   stderr) Try '/nix/store/i3avjvyfgw0ypbb5sq7w5f5d6xhyy586-nix-eval-jobs-2.9.0/bin/nix-eval-jobs --help' for more information.
[ERROR]  failure) Child process exited with error code: 1
[ERROR] Failed to complete requested operation - Last 1 lines of logs:
[ERROR]  failure) Child process exited with error code: 1
[ERROR] -----
[ERROR] Operation failed with error: Child process exited with error code: 1
07:41:19
@whentze:matrix.orgWanja Hentze
In reply to @yuka:yuka.dev
feels like eval times are also better with that
for use with colmena? or as a new thing?
08:07:37
@whentze:matrix.orgWanja HentzeFWIW I've been yearning for this for a while now08:08:01
@yuka:yuka.dev@yuka:yuka.devAs I have said before, colmena is really two separate things: The hive nix expression, and the cli08:08:12
@whentze:matrix.orgWanja Hentzeright08:08:40
@yuka:yuka.dev@yuka:yuka.devSo I see no reason not to use a modules-based new hive expression together with the same old cli08:08:54
@yuka:yuka.dev@yuka:yuka.dev * So I see no reason not to use a modules-based new hive expression together with the same old colmena cli08:08:56
@yuka:yuka.dev@yuka:yuka.devin fact, I just built a colmena-compat module for my hive module system08:09:20
@whentze:matrix.orgWanja Hentzenice08:10:26
@yuka:yuka.dev@yuka:yuka.devI should probably throw these things in a repo somewhere, but eeeeh08:10:55
@whentze:matrix.orgWanja Hentzemore cleanly separating the two halves of colmena might be a thing in itself08:13:42
@whentze:matrix.orgWanja Hentze * more cleanly separating the two halves of colmena might be a worthwhile thing to do08:13:52
@yuka:yuka.dev@yuka:yuka.devThere is more than two halves to it actually08:14:00
@yuka:yuka.dev@yuka:yuka.devthis has improved a lot already recently08:14:40
@whentze:matrix.orgWanja Hentzecare to explain?08:15:07
@yuka:yuka.dev@yuka:yuka.dev

there are several representations that get produced along the way:

  • your colmena hive (the original representation, what you would put in hive.nix with meta and your hosts as attributes)
  • somewhere in between there is also a nixos module system to eval just the meta section (let it have defaults, etc.)
  • nodes attrset (which is very similar to nixosConfigurations) + some specific meta attributes
  • the colmenaEval attrset, which is what the colmena cli needs to do its job

now all these transitions between the representation are currently quite entangled into one big nix function that goes all the way from your colmena hive to colmenaEval

what has improved lately is, that you can decide to call this makeHive function yourself and provide colmena a pre-processed colmenaEval attrset

this also makes it possible to write other functions that produce the same format, but from a different source format

08:27:04
@yuka:yuka.dev@yuka:yuka.dev

and now we can write other useful functions to transform between these representations:

  • pre-existing nixosConfigurations + nixos module system for metaOptions -> colmenaEval ; with this we can use the colmena cli together with a pre-existing nixosConfigurations definition. use colmena cli without the hive concept, or with a different hive mechanism
  • colmena hive -> nixosConfigurations ; use colmena hives with nixos-rebuild, very simple to implement with makeHive

and so on

08:36:59
@yuka:yuka.dev@yuka:yuka.dev *

there are several representations that get produced along the way:

  • your colmena hive (the original representation, what you would put in hive.nix with meta, defaults and your hosts as attributes)
  • somewhere in between there is also a nixos module system to eval just the meta section (give it type checking, default values, etc.)
  • nodes attrset (which is very similar to nixosConfigurations) + some specific meta attributes
  • the colmenaEval attrset, which is what the colmena cli needs to do its job

now all these transitions between the representation are currently quite entangled into one big nix function that goes all the way from your colmena hive to colmenaEval

what has improved lately is, that you can decide to call this makeHive function yourself and provide colmena a pre-processed colmenaEval attrset

this also makes it possible to write other functions that produce the same format, but from a different source format

08:43:42
@whentze:matrix.orgWanja Hentze oh yes, makeHive is a major improvement already 11:31:15
@yuka:yuka.dev@yuka:yuka.dev Yes, but colmenaEval is really just nixosConfigurations in a trench coat, and so conversion functions from/to nixosConfigurations could be useful 11:52:20
@yuka:yuka.dev@yuka:yuka.devso there is still room for improvement11:52:27
@whentze:matrix.orgWanja HentzeI keep wanting a multi-host service abstraction11:52:57
@whentze:matrix.orgWanja Hentzewe have a few of these strung about in an ad-hoc manner in this codebase11:53:07
@whentze:matrix.orgWanja Hentzebut they're all bad in different ways11:53:14
@yuka:yuka.dev@yuka:yuka.devyou can already kinda do that with hive by accessing the nodes specialArgs attribute11:53:19
@whentze:matrix.orgWanja Hentzeyes, we have that11:53:28
@whentze:matrix.orgWanja Hentzeand some ad-hoc conventions on top11:53:33
@whentze:matrix.orgWanja Hentzesometimes we filter nodes by name, other times by tag etc.11:53:55
@yuka:yuka.dev@yuka:yuka.dev

I also generate a groups module arg:

  _module.args.groups = let
    tagNames = lib.unique (
      lib.concatLists (
        lib.mapAttrsToList (
          name: host: host.config.deployment.tags
        ) args.nodes
      )
    );
  in lib.genAttrs tagNames (
    tagName: lib.filter (
      host: lib.elem tagName host.config.deployment.tags
    ) (
      lib.attrValues args.nodes
    )
  );

11:54:09
@whentze:matrix.orgWanja Hentzeo nice11:54:17

Show newer messages


Back to Room ListRoom Version: 6