Sender | Message | Time |
---|---|---|
1 Apr 2025 | ||
10:00:44 | ||
23:15:44 | ||
2 Apr 2025 | ||
07:31:21 | ||
07:32:04 | ||
3 Apr 2025 | ||
01:06:26 | ||
05:48:53 | ||
06:35:10 | ||
11:41:53 | ||
4 Apr 2025 | ||
06:13:17 | ||
5 Apr 2025 | ||
13:06:53 | ||
15:36:30 | ||
6 Apr 2025 | ||
23:26:44 | ||
8 Apr 2025 | ||
01:21:12 | ||
01:59:11 | ||
Hello @roberth, I am working with #clan:clan.lol on trying to decouple the vars proposal by @lassulus:lassul.us from clan, and I have been trying to think about how vars could be implemented in NixOps4. The part about the CRUD operations on some kind of vars (secrets) storage provider feels straightforward. OTOH, the part about interacting with the user for input (the prompts part of the vars interface), and then feeding this input (with maybe previously generated values, see dependencies in the vars interface), into a script which outputs feed into some vars/secrets storage provider, is less straightforward, could you help me understand how that could fit in the NixOps4 architecture? FWIW in point 3.2 of the document linked in this clan PR, I go over how this is currently implemented in clan. PS: I am happy to move this conversation in e.g. discourse. | 02:56:10 | |
* Hello Robert Hensing (roberth), I am working with #clan:clan.lol on trying to decouple the vars proposal by @lassulus:lassul.us from clan, and I have been trying to think about how vars could be implemented in NixOps4. The part about the CRUD operations on some kind of vars (secrets) storage provider feels straightforward. OTOH, the part about interacting with the user for input (the prompts part of the vars interface), and then feeding this input (with maybe previously generated values, see dependencies in the vars interface), into a script which outputs feed into some vars/secrets storage provider, is less straightforward, could you help me understand how that could fit in the NixOps4 architecture? FWIW in point 3.2 of the document linked in this clan PR, I go over how this is currently implemented in clan. PS: I am happy to move this conversation in e.g. discourse. | 02:56:22 | |
06:14:37 | ||
Hi kalessin, thanks for reaching out. If I understand correctly, the vars proposal focuses on wiring up the data flow between scripts inside a NixOS machine, whereas NixOps4 does something similar for expressions while evaluating a whole deployment. It seems that these can indeed be connected. nixops4-nixos is implemented by generating a script that copies the toplevel and switches to it over SSH. This could be extended to write a few files based on the values that are available in the NixOps4 evaluation, such as identifiers and credentials that are generated by other resources. User input isn't part of the design yet, but could be added. I can also imagine that these prompts would be completed with NixOps4 resource data instead of actual interactive user input. | 09:58:47 | |
Maybe I'm a bit off here. I'd have to play around with the PR a bit to get a feel for it. Maybe there's something I'm missing. For now my focus is on making stateful, Terraform-style resources work (i.e. making it remember what previous outputs were so that it can actually let resource providers talk to APIs in a sensible way). So Terraform is a pretty good analog for now, in terms of what it will be able to do - just imagine it's Nix instead of HCL so it's easier to transfer info into a NixOS configuration, and unlike in Terranix, you can also refer to real resource values in the Nix language. | 10:05:58 | |
This might be somewhat expensive, as you're loading a whole NixOS configuration just to access the vars definitions | 10:09:29 | |
(broad adoption of modular services could drive down that evaluation cost, 2-4× for system.build.toplevel , maybe more for evaluating a small part of a config?) | 10:12:59 | |
17:43:33 | ||
11:24:05 | ||
Thank you for your followup Robert Hensing (roberth)!
I think prompts could actually be completed with both/either NixOps4 resources data and/or user input.
@lassulus:lassul.us, made a separate repo, it's a good playground, and we gotta fix the test…
While I haven't used Terraform extensively, I feel like I have a good grasp over it. I think the CRUD operations on some secrets storage backend already maps well to the Nixops4 model. The secret generation with the prompts, their outputs going into the script that generate the secrets, seems to be the tricky part, is that why you mentioned focus?
It is a little expensive, I know the clan team has been trying to bring the cost down, and do some caching. I am not sure I understand how modular services would reduce evaluation time? Would it help avoid loading the entire NixOS module library, which I currently understand is one of the main issues regarding NixOS configurations eval times (and RAM usage)? | 17:50:57 | |
Loading all the services is a fairly significant portion of NixOS evaluation. Each file, a bunch of options and each config = mkIf cfg.enable has to be evaluated, at least. | 18:08:34 | |
Ok, so it's not just all of the imports phase? | 18:10:23 | |
Seems safe to assume the list hasn't shrunk in the past 3.5 years :) | 18:18:07 | |
I assumed vars generation would run on the host, being part of the config. Is this representative or accidentally wonky? https://github.com/Lassulus/vars/blob/066dbc738838b79be6bc80b88559cb88e0fffffd/testing.nix#L75 | 18:25:40 | |
fwiw this way the test can't run on a darwin VM host, but that's probably besides the point | 18:27:50 | |
It's one representation, e.g. in clan vars generation is done from on the developer machine, ahead of deploying to a nixos host. | 18:27:52 |