!atvIbxHoEqNcAIxYpN:nixos.org

NixOS AWS

61 Members
15 Servers

Load older messages

SenderMessageTime
28 Apr 2025
@notmycommit:notwork.indbalan Our scripts do some s3 gets / aws ssm get-parmeters etc to assemble the final nix config. We do have workaround for this to wrap the execution in nix-env. But IMO awscli seems quite useful on an amazon AMI. 11:31:24
@notmycommit:notwork.indbalan

Another option would be to look for nix-shell shebang, like

       #! /usr/bin/env nix-shell
       #! nix-shell -i runghc --packages 'haskellPackages.ghcWithPackages (ps: [ps.download-curl ps.tagsoup])'

in the init script where we determine the type in amazon-init.nix and run it with nix-shell instead of ${pkgs.runtimeShell}

11:51:29
@arianvp:matrix.orgArianwe download a closure and nixos-rebuild switch into ti11:57:01
@arianvp:matrix.orgArianwhy would you want to run imperative scripts if you can just deploy a new NixOS build with the script you want to run? Sounds more robust :D11:58:04
@notmycommit:notwork.indbalanHaha -- we use imperative secrets to assemble a nix config that we can switch to (mostly secrets and some other foo). But this is definitely not a blocker for us or anything.11:59:41
@arianvp:matrix.orgArian That sounds terrible :D 12:00:14
@arianvp:matrix.orgArian

you can also build your own ami. there are instructions in https://github.com/nixos/amis

But I highly advise against building NixOS configs with secrets in them. The nix store is world-readable and not a suitable place for storing any kind of secret.

Secrets should either be avoided (by using IAM roles) or should be a runtime concern and fetched with said IAM roles

12:14:52
@arianvp:matrix.orgAriananything else is bound to cause serious security issues12:15:20
@arianvp:matrix.orgArianAnd I can’t suggest int good faith to go that route12:15:33
@arianvp:matrix.orgArian * 12:15:41
@arianvp:matrix.orgArian *

you can also build your own ami. there are instructions in https://github.com/nixos/amis

But I highly advise against building NixOS configs with secrets in them. The nix store is world-readable and not a suitable place for storing any kind of secret.

Secrets should either be avoided (by using IAM roles) or should be a runtime concern and fetched with said IAM roles from a dedicated secrets manager like SSM parameter store, AWS SecretsManager, or something like OpenBao or Vault

12:16:19
@notmycommit:notwork.indbalanRedacted or Malformed Event14:41:19
@notmycommit:notwork.indbalanYup, We do this weird route more or less for avoiding storing secrets anywhere other than the instance.15:04:12
@notmycommit:notwork.indbalanSecrets are in vault or aws depending on the layer and they get populated on first boot in the config15:04:57
@adam:robins.wtf@adam:robins.wtfI wrote a simple module that will pull a secret down with the CLI given an ARN and some permissions. Creates a basic dir in /run to avoid storing them on disk22:51:20
@adam:robins.wtf@adam:robins.wtfAPI is roughly what you get from agenix22:51:47
@adam:robins.wtf@adam:robins.wtfWe do pull one secret during cloud-init, but otherwise try and keep it as simple as possible. Cloud init's main job is to discover the proper system store path, pull it, and switch to it, and a couple other imperative things about the system for PS1 and an env file. Our apps also now read their secrets directly on startup, so most secrets never get written outside memory.22:53:48
@kranzes:matrix.orgIlan Joselevich (Kranzes)
In reply to @adam:robins.wtf
I wrote a simple module that will pull a secret down with the CLI given an ARN and some permissions. Creates a basic dir in /run to avoid storing them on disk
Do you have this in a public repo somewhere? 		23:04:17
29 Apr 2025
@adam:robins.wtf@adam:robins.wtfNo, it's all internal currently00:07:43
@adam:robins.wtf@adam:robins.wtfI'll try and remember to throw it in a gist at least01:30:26
@adam:robins.wtf@adam:robins.wtfIt's a very simplistic module :)01:30:37
@adam:robins.wtf@adam:robins.wtfhttps://gist.github.com/adamcstephens/9f45c33fa02270f669f2cd164faafea714:21:37
1 May 2025
@commiterate:matrix.orgcommiterate

might be easier to write a systemd unit (e.g. oneshot + optional timer or ExecStartPre) which does the credential pull than relying on boot user scripts.

Ideally the boot userscript just does a nixos-rebuild switch --flake {flake URL} and nothing else

19:46:13
@commiterate:matrix.orgcommiteratee.g. I'm using an ExecStartPre in a GitLab Runner systemd unit which fetches a GitLab Runner authn token from AWS Secrets Manager19:46:58
@commiterate:matrix.orgcommiterate *

might be easier to write a systemd unit (e.g. oneshot + optional timer or ExecStartPre) which does the credential pull ratherer than relying on boot user scripts.

Ideally the boot userscript just does a nixos-rebuild switch --flake {flake URL} and nothing else

19:55:12
@commiterate:matrix.orgcommiterate *

might be easier to write a systemd unit (e.g. oneshot + optional timer or ExecStartPre) which does the credential pull rather than relying on boot user scripts.

Ideally the boot userscript just does a nixos-rebuild switch --flake {flake URL} and nothing else

19:55:18
@adam:robins.wtf@adam:robins.wtfsure, but if you need a secret to access the flake URL then you have to bootstrap somehow :)20:10:34
@adam:robins.wtf@adam:robins.wtfmade me realize i accidentally dropped the service from the above gist when redacting it. updated.20:36:15
3 May 2025
@commiterate:matrix.orgcommiterate🤦 forgot about private flakes, my company still uses a network trust model instead of zero trust so just being in the right VPC is enough19:29:57
4 May 2025
@arianvp:matrix.orgArianDon’t do nix eval on your hosts! just pull nix store paths!09:04:38

Show newer messages

Back to Room ListRoom Version: 10