!UKDpaKNNsBpOPfLWfX:zhaofeng.li

Colmena

326 Members
A simple, stateless NixOS deployment tool - https://github.com/zhaofengli/colmena111 Servers

Load older messages


SenderMessageTime
11 Jul 2022
@jairo:recallstack.icu@jairo:recallstack.iculet me do some tests11:11:58
@schnecfk:ruhr-uni-bochum.deCRTified

So right now I'd imagine a workflow similar to:

  1. Terraform creates the infrastructure
  2. Terraform generates a json output of that infrastructure
  3. nix/colmena consumes it to generate the appropriate config (I'd guess mainly modifying the deployment-keys, while keeping the larger portion of the config in "pure" config files) - this might require impure flake evaluation
  4. colmena deploys the config
11:14:22
@schnecfk:ruhr-uni-bochum.deCRTifiedPlease correct me if that is wrong 😄11:14:32
@jairo:recallstack.icu@jairo:recallstack.icuit should work11:15:00
@jairo:recallstack.icu@jairo:recallstack.icuHow to use impure flake evaluation?11:15:11
@schnecfk:ruhr-uni-bochum.deCRTifiediirc colmena defaults to it11:15:57
@jairo:recallstack.icu@jairo:recallstack.icuah!11:16:05
@schnecfk:ruhr-uni-bochum.deCRTified Otherwise, most nix tools use --impure 11:16:06
@jairo:recallstack.icu@jairo:recallstack.icugood11:16:10
@jairo:recallstack.icu@jairo:recallstack.icuok let me try11:16:13
@schnecfk:ruhr-uni-bochum.deCRTifiedit mainly means that the git tree for the config might contain uncommited changes11:16:32
@schnecfk:ruhr-uni-bochum.deCRTifiedBut I don't know whether the terraform json output should be tracked by git 🤔11:16:49
@schnecfk:ruhr-uni-bochum.deCRTified * But I don't know whether the terraform json output should be tracked by git 🤔 (to make it pure)11:16:57
@jairo:recallstack.icu@jairo:recallstack.icuit shouldn't11:16:58
@jairo:recallstack.icu@jairo:recallstack.icu see, with the same Terraform declarations I can run terraform apply or terraform destroy 11:17:26
@jairo:recallstack.icu@jairo:recallstack.icu

So let's say:

  1. apply. HostA IP = 1.1.1.1
  2. destroy
  3. apply. HostA IP = 1.1.1.2
11:18:05
@jairo:recallstack.icu@jairo:recallstack.icuthat's why Terraform keeps the state outside git (and for other reasons), and why this can't be pure to be usable11:18:28
@jairo:recallstack.icu@jairo:recallstack.icuWell, I could git-commit the terraform output anyway 😆11:18:40
@jairo:recallstack.icu@jairo:recallstack.icuBut direct linking to terraform state seems more useful imho (I might be wrong)11:19:04
@jairo:recallstack.icu@jairo:recallstack.icu

well... just like I was expecting, it doesn't work. 😕
In my flake, I have:

let inventory = builtins.fromJSON (
      builtins.readFile
      (
        pkgs.runCommandLocal "terraform-inventory" {
          src = ./.;
        } ''
          export PATH=${self.outputs.devShells.${pkgs.system}.default}/bin:$PATH
          bash $src/ansible/inventories/terraform.sh > $out
        ''
      )
    );
in colmena = { ... } # Stuff that parses the inventory and generates machine info

The results, as I was expecting:

> colmena build
warning: Git tree '/var/home/yajo/prodevel/moduground' is dirty
[INFO ] Using flake: git+file:///var/home/yajo/prodevel/moduground
[INFO ] Enumerating nodes...
warning: Git tree '/var/home/yajo/prodevel/moduground' is dirty
building '/nix/store/pa4n1x0ncgw669zyksz89kwflpr51nl8-terraform-inventory.drv'...
â•·
│ Error: Backend initialization required, please run "terraform init"
│ 
│ Reason: Initial configuration of the requested backend "http"
[...]

Basically it means that it's trying to build that inventory as a normal derivation, in isolation. Thus, it doesn't get the necessary environmental secrets that Terraform uses to access its state, and fails.

The dynamic inventory should be evaluated outside of the nix expression, and maybe passed in as an argument. But my guess is that this would be common enough so as to colmena needing to add support for it out of the box somehow, right?

12:34:06
@buckley310:matrix.orgBuckleySo wait, is there a pure evaluation mode?13:54:48
@buckley310:matrix.orgBuckleyThat sounds nice13:55:02
@linus:schreibt.jetztLinux Hackermannot for colmena AFAIK13:55:18
@jairo:recallstack.icu@jairo:recallstack.icuI'm using flakes, that's pure by default AFAIK 13:56:59
@linus:schreibt.jetztLinux Hackermannot with colmena13:57:10
@linus:schreibt.jetztLinux Hackermancolmena isn't currently able to evaluate purely13:57:26
@linus:schreibt.jetztLinux Hackermanhttps://github.com/zhaofengli/colmena/blob/ac8852dcdf7ab6e9524600518414a63dbfb115b1/src/nix/evaluator/nix_eval_jobs.rs#L8513:57:55
@jairo:recallstack.icu@jairo:recallstack.icuBut then how to get json output from one script into it as an argument? 🤔14:01:07
14 Jul 2022
@ty:tjll.nettylerjl joined the room.21:26:34
@ty:tjll.nettylerjl Hey hey - is there a good way to consume a nixosConfiguration from something like a flake output to feed into colmena? I'd love to pull a host's config out of something like my flake that uses digga/devos and feed it into colmena to apply to the host 21:29:38

Show newer messages


Back to Room ListRoom Version: 6