!UKDpaKNNsBpOPfLWfX:zhaofeng.li

Colmena

253 Members
A simple, stateless NixOS deployment tool - https://github.com/zhaofengli/colmena87 Servers

Load older messages

SenderMessageTime
16 Nov 2024
@rcouto:matrix.org@rcouto:matrix.org joined the room.21:35:13
@inomena:matrix.orginomena joined the room.22:45:18
@inomena:matrix.orginomena hi, looking at the docs w/flakes each host is defined in outputs.colmena.<hostname> similar to nixosConfigurations, but is it possible to instead read each host's configuration from separate files?
e.g. instead of outputs.colmena.host-a and outputs.colmena.host-b I could refer to files hosts/host-a.nix and hosts/host-b.nix (and all possible hosts within the hosts/ directory without specifying each one in flake.nix) 		22:49:07
@inomena:matrix.orginomena I would still want to use outputs.colmena.meta and outputs.colmena.defaults in this scenario 22:51:13
17 Nov 2024
@inomena:matrix.orginomena

think I figured it out, or atleast it seems to work:

outputs = {nixpkgs, ...}:
  let
    hosts = nixpkgs.lib.mapAttrs'
      (filename: _: {
        name = nixpkgs.lib.nameFromURL filename ".";
        value = [ ./hosts/${filename} ];
      })
      (builtins.readDir ./hosts);
  in {
    colmena = hosts // {
      # ...
    };
};
08:39:26
@ibizaman:matrix.orgibizaman
In reply to @inomena:matrix.org

think I figured it out, or atleast it seems to work:

outputs = {nixpkgs, ...}:
  let
    hosts = nixpkgs.lib.mapAttrs'
      (filename: _: {
        name = nixpkgs.lib.nameFromURL filename ".";
        value = [ ./hosts/${filename} ];
      })
      (builtins.readDir ./hosts);
  in {
    colmena = hosts // {
      # ...
    };
};
Oh nice you went the extra mile with readDir. I was going to suggest plain imports but that works well! 		08:57:01
@inomena:matrix.orginomenaI wanted it to be fully dynamic, so that the only thing I need to do to add a new host to the hive is to define a single <hostname>.nix file in the hosts dir and nothing else08:57:56
@inomena:matrix.orginomenanot having to also define it in a few other places08:58:09
@inomena:matrix.orginomena * not having to also define imports in a few other places08:58:18
@nixy:nixy.moenixy
In reply to @ser:sergevictor.eu
can colmena secrets deploy a whole directory, not single files only?
I don't believe so. But you can use the keyCommand option to create an archive (tar, zip) from a directory and then use systemd services to extract it. That's what I do for this at least 		14:06:26
@ser:sergevictor.euser(ial)that's a nice concept, i didn't think about it, thanks!14:08:58
@jh-devv:matrix.org@jh-devv:matrix.org joined the room.18:39:43
@jh-devv:matrix.org@jh-devv:matrix.orgWhat tool do you people think would fit well with Colmena when deploying bare metal?18:50:44
@jh-devv:matrix.org@jh-devv:matrix.orgUnattended install etc.18:50:52
18 Nov 2024
@luna:bddvlpr.comLuna
In reply to@ibizaman:matrix.org
I did the opposite for some of my machines recently to test deploy-rs and had no issue. Feel free to post the log here next time.
Finally got to it, here's what I get:
[INFO ] Enumerating nodes...
error:
       … while calling the 'attrNames' builtin
         at «string»:1:190:
            1| with builtins; let assets = getFlake "path:/tmp/colmena-assets-HeCr3l?lastModified=1731921370&narHash=sha256-xWcw5WYfWGtmkZGFW7NjzxlGh8vDejJzDNJj2nJg7Dk%3D"; hive = assets.processFlake; in attrNames hive.nodes
             |                                                                                                                                                                                              ^

       … while calling the 'getFlake' builtin
         at «string»:1:29:
            1| with builtins; let assets = getFlake "path:/tmp/colmena-assets-HeCr3l?lastModified=1731921370&narHash=sha256-xWcw5WYfWGtmkZGFW7NjzxlGh8vDejJzDNJj2nJg7Dk%3D"; hive = assets.processFlake; in attrNames hive.nodes
             |                             ^

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: cannot update unlocked flake input 'hive' in pure mode
[ERROR] -----
[ERROR] Operation failed with error: Child process exited with error code: 1
Hint: Backtrace available - Use `RUST_BACKTRACE=1` environment variable to display a backtrace
09:17:57
@luna:bddvlpr.comLuna Oh my bad, I forgot --experimental-flake-eval, this is the log I get:
[WARN ] Using direct flake evaluation (experimental)
[INFO ] Enumerating nodes...
[INFO ] Selected all 4 nodes.
       ❌ 4s Failed: Child process exited with error code: 1
 (...) ❌ 4s Evaluation failed: Child process exited with error code: 1                                           
[ERROR] Failed to complete requested operation - Last 1 lines of logs:
[ERROR]  failure) Child process exited with error code: 1
[ERROR] Failed to evaluate phobos, ceres, deimos, and ariel - Last 20 lines of logs:
[ERROR]   stderr)          1575|         else
[ERROR]   stderr) 
[ERROR]   stderr)        … while evaluating the attribute 'value'
[ERROR]   stderr)          at /nix/store/hfz1qqd0z8amlgn8qwich1dvkmldik36-source/lib/modules.nix:816:9:
[ERROR]   stderr)           815|     in warnDeprecation opt //
[ERROR]   stderr)           816|       { value = addErrorContext "while evaluating the option `${showOption loc}':" value;
[ERROR]   stderr)              |         ^
[ERROR]   stderr)           817|         inherit (res.defsFinal') highestPrio;
[ERROR]   stderr) 
[ERROR]   stderr)        … while evaluating the option `system.build.toplevel':
[ERROR]   stderr) 
[ERROR]   stderr)        … while evaluating definitions from `/nix/store/hfz1qqd0z8amlgn8qwich1dvkmldik36-source/nixos/modules/system/activation/top-level.nix':
[ERROR]   stderr) 
[ERROR]   stderr)        (stack trace truncated; use '--show-trace' to show the full, detailed trace)
[ERROR]   stderr) 
[ERROR]   stderr)        error:
[ERROR]   stderr)        Failed assertions:
[ERROR]   stderr)        - The ‘fileSystems’ option does not specify your root file system.
[ERROR]   stderr)        - You must set the option ‘boot.loader.grub.devices’ or 'boot.loader.grub.mirroredBoots' to make the system bootable.
[ERROR]  failure) Child process exited with error code: 1
[ERROR] -----
[ERROR] Operation failed with error: Child process exited with error code: 1
Hint: Backtrace available - Use `RUST_BACKTRACE=1` environment variable to display a backtrace
Is disko perhaps not compatible with colmena? 		09:20:22
@ibizaman:matrix.orgibizaman
In reply to @jh-devv:matrix.org
What tool do you people think would fit well with Colmena when deploying bare metal?
Any tool should be compatible. I used nixos-anywhere successfully and heard about nixos-infect working too. I’ve got a flake template for the former https://github.com/ibizaman/skarabox if you want some inspiration. 		09:27:06
@luna:bddvlpr.comLuna Perhaps its not reading any of the configuration as the log shows
 (...) | evaluation warning: system.stateVersion is not set, defaulting to 25.05. Read why this matters on https://nixos.org/manual/nixos/stable/options.html#opt-system.stateVersion.
While a repl shows
nix-repl> outputs.nixosConfigurations.ariel.config.system.stateVersion
"24.05"

nix-repl>
09:27:29
@ibizaman:matrix.orgibizaman
In reply to @luna:bddvlpr.com
Oh my bad, I forgot --experimental-flake-eval, this is the log I get:
[WARN ] Using direct flake evaluation (experimental)
[INFO ] Enumerating nodes...
[INFO ] Selected all 4 nodes.
       ❌ 4s Failed: Child process exited with error code: 1
 (...) ❌ 4s Evaluation failed: Child process exited with error code: 1                                           
[ERROR] Failed to complete requested operation - Last 1 lines of logs:
[ERROR]  failure) Child process exited with error code: 1
[ERROR] Failed to evaluate phobos, ceres, deimos, and ariel - Last 20 lines of logs:
[ERROR]   stderr)          1575|         else
[ERROR]   stderr) 
[ERROR]   stderr)        … while evaluating the attribute 'value'
[ERROR]   stderr)          at /nix/store/hfz1qqd0z8amlgn8qwich1dvkmldik36-source/lib/modules.nix:816:9:
[ERROR]   stderr)           815|     in warnDeprecation opt //
[ERROR]   stderr)           816|       { value = addErrorContext "while evaluating the option `${showOption loc}':" value;
[ERROR]   stderr)              |         ^
[ERROR]   stderr)           817|         inherit (res.defsFinal') highestPrio;
[ERROR]   stderr) 
[ERROR]   stderr)        … while evaluating the option `system.build.toplevel':
[ERROR]   stderr) 
[ERROR]   stderr)        … while evaluating definitions from `/nix/store/hfz1qqd0z8amlgn8qwich1dvkmldik36-source/nixos/modules/system/activation/top-level.nix':
[ERROR]   stderr) 
[ERROR]   stderr)        (stack trace truncated; use '--show-trace' to show the full, detailed trace)
[ERROR]   stderr) 
[ERROR]   stderr)        error:
[ERROR]   stderr)        Failed assertions:
[ERROR]   stderr)        - The ‘fileSystems’ option does not specify your root file system.
[ERROR]   stderr)        - You must set the option ‘boot.loader.grub.devices’ or 'boot.loader.grub.mirroredBoots' to make the system bootable.
[ERROR]  failure) Child process exited with error code: 1
[ERROR] -----
[ERROR] Operation failed with error: Child process exited with error code: 1
Hint: Backtrace available - Use `RUST_BACKTRACE=1` environment variable to display a backtrace
Is disko perhaps not compatible with colmena?
There are two errors at the end there, setting root filesystem and grub stuff. Usually those come from running the hardware generator on the new machine and that gets you a hardware-configuration.nix file. You must include that file or recreate the options it is setting for the configuration to be happy. 		09:29:54
@jh-devv:matrix.org@jh-devv:matrix.org
In reply to @ibizaman:matrix.org
Any tool should be compatible. I used nixos-anywhere successfully and heard about nixos-infect working too. I’ve got a flake template for the former https://github.com/ibizaman/skarabox if you want some inspiration.
Does this support iPXE? 		09:30:06
@luna:bddvlpr.comLuna
In reply to@ibizaman:matrix.org
There are two errors at the end there, setting root filesystem and grub stuff. Usually those come from running the hardware generator on the new machine and that gets you a hardware-configuration.nix file. You must include that file or recreate the options it is setting for the configuration to be happy.
Those are correctly set and can be seen through the repl. Maybe I'm misunderstanding, does it (like deploy-rs) check for configurations at outputs.nixosConfigurations for matching attributes to outputs.colmena? 		09:31:28
@luna:bddvlpr.comLuna 
{
  self,
  inputs,
  ...
}: let
  inherit (self) outputs;
  inherit (builtins) mapAttrs;
in {
  flake = {
    colmena =
      (mapAttrs (host: value: {
          deployment = {
            targetHost = "${host}.cloud.bddvlpr.com";
          };
        })
        outputs.nixosConfigurations)
      // {
        meta = {
          nixpkgs = import inputs.nixpkgs {
            system = "x86_64-linux";
            overlays = [];
          };
        };
      };

    colmenaHive = inputs.colmena.lib.makeHive outputs.colmena;
  };
}
This is the current configuration, btw. 		09:33:32
@ibizaman:matrix.orgibizaman
In reply to @jh-devv:matrix.org
Does this support iPXE?
Ah no I didn’t do that in my template. But it’s definitely doable. I found a few links though https://wiki.nixos.org/wiki/Netboot and https://carlosvaz.com/posts/ipxe-booting-with-nixos/ 		09:33:34
@ibizaman:matrix.orgibizaman
In reply to @luna:bddvlpr.com
{
  self,
  inputs,
  ...
}: let
  inherit (self) outputs;
  inherit (builtins) mapAttrs;
in {
  flake = {
    colmena =
      (mapAttrs (host: value: {
          deployment = {
            targetHost = "${host}.cloud.bddvlpr.com";
          };
        })
        outputs.nixosConfigurations)
      // {
        meta = {
          nixpkgs = import inputs.nixpkgs {
            system = "x86_64-linux";
            overlays = [];
          };
        };
      };

    colmenaHive = inputs.colmena.lib.makeHive outputs.colmena;
  };
}
This is the current configuration, btw.
I’m maybe misreading, did you leave out the configuration for the host you’re trying the build the config for? 		09:36:43
@luna:bddvlpr.comLuna No, I think I get what I'm doing wrong now. I thought that it just required outputs.colmena to get the deployment settings and would actually look up the host at outputs.nixosConfigurations like deploy-rs would. I'll try just merging them now. 09:37:45
@ibizaman:matrix.orgibizaman
In reply to @luna:bddvlpr.com
No, I think I get what I'm doing wrong now. I thought that it just required outputs.colmena to get the deployment settings and would actually look up the host at outputs.nixosConfigurations like deploy-rs would. I'll try just merging them now.
Ah I see indeed that seems like the likely culprit. 		09:38:17
@luna:bddvlpr.comLunaAs for the meta nixpkgs, does this have to be a set architecture?09:39:03
@luna:bddvlpr.comLunaCan't it be dynamic depending on the host running commands?09:39:16
@ibizaman:matrix.orgibizaman
In reply to @luna:bddvlpr.com
Can't it be dynamic depending on the host running commands?
Dynamic, not sure. But you can use https://colmena.cli.rs/unstable/reference/meta.html#nodenixpkgs for node specific nixpkgs. There’s an example here https://colmena.cli.rs/unstable/tutorial/index.html 		09:57:42
@luna:bddvlpr.comLunaFinally got it evaluating with https://github.com/zhaofengli/colmena/issues/60#issuecomment-1510496861. I still don't get the use of meta.nixpkgs in particular though.10:01:02

Show newer messages

Back to Room ListRoom Version: 6