!BgJZHVOYkwVcEKLAyM:nixos.org

NixOS Deployments

1106 Members
NixOS Deployment tooling277 Servers

Load older messages


SenderMessageTime
1 Apr 2025
@zapotecorum:matrix.orgzapotecorum joined the room.10:00:44
@adam_neverwas:matrix.orgAdam Neverwas set a profile picture.23:15:44
2 Apr 2025
@fabaff:matrix.orgfabaff left the room.07:31:21
@fabaff:matrix.orgfabaff joined the room.07:32:04
3 Apr 2025
@diamondburned:matrix.orgdiamond (it/its) changed their display name from Diamond (it/she) to diamond (it/its).01:06:26
@andrewm:element.ioAndrew Morgan (anoa) {he/him} changed their display name from Andrew Morgan (anoa) {he/him} [away; til 27th] to Andrew Morgan (anoa) {he/him} [back Apr 14].05:48:53
@andrewm:element.ioAndrew Morgan (anoa) {he/him} changed their display name from Andrew Morgan (anoa) {he/him} [back Apr 14] to Andrew Morgan (anoa) {he/him} [back Apr 14].06:35:10
@mjolnir:nixos.orgNixOS Moderation Bot banned @cafkafk:fem.gg@cafkafk:fem.gg (<no reason supplied>).11:41:53
4 Apr 2025
@mjolnir:nixos.orgNixOS Moderation Bot unbanned @cafkafk:fem.gg@cafkafk:fem.gg.06:13:17
5 Apr 2025
@zexu:matrix.orgzexu joined the room.13:06:53
@tinybronca:sibnsk.net@tinybronca:sibnsk.net left the room.15:36:30
6 Apr 2025
@bashfulrobot.:matrix.orgbashfulrobot changed their profile picture.23:26:44
8 Apr 2025
@kalessin:cybre.spacekalessin joined the room.01:21:12
@delatorre:matrix.orgEmerson Delatorre joined the room.01:59:11
@kalessin:cybre.spacekalessin Hello @roberth, I am working with #clan:clan.lol on trying to decouple the vars proposal by @lassulus:lassul.us from clan, and I have been trying to think about how vars could be implemented in NixOps4. The part about the CRUD operations on some kind of vars (secrets) storage provider feels straightforward. OTOH, the part about interacting with the user for input (the prompts part of the vars interface), and then feeding this input (with maybe previously generated values, see dependencies in the vars interface), into a script which outputs feed into some vars/secrets storage provider, is less straightforward, could you help me understand how that could fit in the NixOps4 architecture? FWIW in point 3.2 of the document linked in this clan PR, I go over how this is currently implemented in clan. PS: I am happy to move this conversation in e.g. discourse. 02:56:10
@kalessin:cybre.spacekalessin * Hello Robert Hensing (roberth), I am working with #clan:clan.lol on trying to decouple the vars proposal by @lassulus:lassul.us from clan, and I have been trying to think about how vars could be implemented in NixOps4. The part about the CRUD operations on some kind of vars (secrets) storage provider feels straightforward. OTOH, the part about interacting with the user for input (the prompts part of the vars interface), and then feeding this input (with maybe previously generated values, see dependencies in the vars interface), into a script which outputs feed into some vars/secrets storage provider, is less straightforward, could you help me understand how that could fit in the NixOps4 architecture? FWIW in point 3.2 of the document linked in this clan PR, I go over how this is currently implemented in clan. PS: I am happy to move this conversation in e.g. discourse. 02:56:22
@emin017:matrix.orgemin017 joined the room.06:14:37
@roberthensing:matrix.orgRobert Hensing (roberth) Hi kalessin, thanks for reaching out. If I understand correctly, the vars proposal focuses on wiring up the data flow between scripts inside a NixOS machine, whereas NixOps4 does something similar for expressions while evaluating a whole deployment. It seems that these can indeed be connected. nixops4-nixos is implemented by generating a script that copies the toplevel and switches to it over SSH. This could be extended to write a few files based on the values that are available in the NixOps4 evaluation, such as identifiers and credentials that are generated by other resources. User input isn't part of the design yet, but could be added. I can also imagine that these prompts would be completed with NixOps4 resource data instead of actual interactive user input. 09:58:47
@roberthensing:matrix.orgRobert Hensing (roberth)Maybe I'm a bit off here. I'd have to play around with the PR a bit to get a feel for it. Maybe there's something I'm missing. For now my focus is on making stateful, Terraform-style resources work (i.e. making it remember what previous outputs were so that it can actually let resource providers talk to APIs in a sensible way). So Terraform is a pretty good analog for now, in terms of what it will be able to do - just imagine it's Nix instead of HCL so it's easier to transfer info into a NixOS configuration, and unlike in Terranix, you can also refer to real resource values in the Nix language.10:05:58
@roberthensing:matrix.orgRobert Hensing (roberth)
 1. The NixOS configuration [specifies] which vars backend to use;
 1. The vars definitions are [exported] to [JSON];

This might be somewhat expensive, as you're loading a whole NixOS configuration just to access the vars definitions

10:09:29
@roberthensing:matrix.orgRobert Hensing (roberth) (broad adoption of modular services could drive down that evaluation cost, 2-4× for system.build.toplevel, maybe more for evaluating a small part of a config?) 10:12:59
@lassulus:lassul.uslassulus joined the room.17:43:33
@yuri:peori.space@yuri:peori.space left the room.11:24:05
@kalessin:cybre.spacekalessin

Thank you for your followup Robert Hensing (roberth)!

User input isn't part of the design yet, but could be added. I can also imagine that these prompts would be completed with NixOps4 resource data instead of actual interactive user input.

I think prompts could actually be completed with both/either NixOps4 resources data and/or user input.

Maybe I'm a bit off here. I'd have to play around with the PR a bit to get a feel for it. Maybe there's something I'm missing.

@lassulus:lassul.us, made a separate repo, it's a good playground, and we gotta fix the test…

For now my focus is on making stateful, Terraform-style resources work (i.e. making it remember what previous outputs were so that it can actually let resource providers talk to APIs in a sensible way). So Terraform is a pretty good analog for now, in terms of what it will be able to do - just imagine it's Nix instead of HCL so it's easier to transfer info into a NixOS configuration, and unlike in Terranix, you can also refer to real resource values in the Nix language.

While I haven't used Terraform extensively, I feel like I have a good grasp over it. I think the CRUD operations on some secrets storage backend already maps well to the Nixops4 model. The secret generation with the prompts, their outputs going into the script that generate the secrets, seems to be the tricky part, is that why you mentioned focus?

  1. The NixOS configuration [specifies] which vars backend to use;
  2. The vars definitions are [exported] to [JSON];

This might be somewhat expensive, as you're loading a whole NixOS configuration just to access the vars definitions. (broad adoption of modular services could drive down that evaluation cost, 2-4× for system.build.toplevel, maybe more for evaluating a small part of a config?)

It is a little expensive, I know the clan team has been trying to bring the cost down, and do some caching. I am not sure I understand how modular services would reduce evaluation time? Would it help avoid loading the entire NixOS module library, which I currently understand is one of the main issues regarding NixOS configurations eval times (and RAM usage)?

17:50:57
@roberthensing:matrix.orgRobert Hensing (roberth) Loading all the services is a fairly significant portion of NixOS evaluation. Each file, a bunch of options and each config = mkIf cfg.enable has to be evaluated, at least. 18:08:34
@kalessin:cybre.spacekalessin Ok, so it's not just all of the imports phase? 18:10:23
@roberthensing:matrix.orgRobert Hensing (roberth)

But to really see the potential:

  1. Make significant cuts in modules-list.nix and fix a few broken options references
  2. Watch instantiation of toplevel go down from 4.0 to 1.4 seconds.
  • https://github.com/NixOS/nixpkgs/issues/137168

Seems safe to assume the list hasn't shrunk in the past 3.5 years :)

18:18:07
@roberthensing:matrix.orgRobert Hensing (roberth)I assumed vars generation would run on the host, being part of the config. Is this representative or accidentally wonky? https://github.com/Lassulus/vars/blob/066dbc738838b79be6bc80b88559cb88e0fffffd/testing.nix#L7518:25:40
@roberthensing:matrix.orgRobert Hensing (roberth)fwiw this way the test can't run on a darwin VM host, but that's probably besides the point18:27:50
@kalessin:cybre.spacekalessinIt's one representation, e.g. in clan vars generation is done from on the developer machine, ahead of deploying to a nixos host.18:27:52

Show newer messages


Back to Room ListRoom Version: 6