!atvIbxHoEqNcAIxYpN:nixos.org

NixOS AWS

61 Members
15 Servers

Load older messages


SenderMessageTime
13 Apr 2025
@arianvp:matrix.orgArianDecided that dealing with MacOS in EC2 is not worth my time09:58:45
@arianvp:matrix.orgArianEspecially with how it interacts with nix (can't install nix on the ebs volume without clicking buttons in the UI)09:59:05
@commiterate:matrix.orgcommiterateit's pretty much unusable, though most of the blame is on Apple's EULA and TCC/FDA19:54:22
28 Apr 2025
@notmycommit:notwork.indbalanHey y'all not sure if this is documented somewhere, whats the reason AWS AMIs does not install aws cli by default? Is the expectation that we should bring it in with userdata?11:05:20
@arianvp:matrix.orgArian It’s not part of the amazon-image.nix profile. We could add it if you want? but yeh usually people just reconfigure their NixOS machines through user-data after boot 11:23:15
@arianvp:matrix.orgArianwhy do you need it?11:23:25
@notmycommit:notwork.indbalan Our scripts do some s3 gets / aws ssm get-parmeters etc to assemble the final nix config. We do have workaround for this to wrap the execution in nix-env. But IMO awscli seems quite useful on an amazon AMI. 11:31:24
@notmycommit:notwork.indbalan

Another option would be to look for nix-shell shebang, like

       #! /usr/bin/env nix-shell
       #! nix-shell -i runghc --packages 'haskellPackages.ghcWithPackages (ps: [ps.download-curl ps.tagsoup])'

in the init script where we determine the type in amazon-init.nix and run it with nix-shell instead of ${pkgs.runtimeShell}

11:51:29
@arianvp:matrix.orgArianwe download a closure and nixos-rebuild switch into ti11:57:01
@arianvp:matrix.orgArianwhy would you want to run imperative scripts if you can just deploy a new NixOS build with the script you want to run? Sounds more robust :D11:58:04
@notmycommit:notwork.indbalanHaha -- we use imperative secrets to assemble a nix config that we can switch to (mostly secrets and some other foo). But this is definitely not a blocker for us or anything.11:59:41
@arianvp:matrix.orgArian That sounds terrible :D 12:00:14
@arianvp:matrix.orgArian

you can also build your own ami. there are instructions in https://github.com/nixos/amis

But I highly advise against building NixOS configs with secrets in them. The nix store is world-readable and not a suitable place for storing any kind of secret.

Secrets should either be avoided (by using IAM roles) or should be a runtime concern and fetched with said IAM roles

12:14:52
@arianvp:matrix.orgAriananything else is bound to cause serious security issues12:15:20
@arianvp:matrix.orgArianAnd I can’t suggest int good faith to go that route12:15:33
@arianvp:matrix.orgArian * 12:15:41
@arianvp:matrix.orgArian *

you can also build your own ami. there are instructions in https://github.com/nixos/amis

But I highly advise against building NixOS configs with secrets in them. The nix store is world-readable and not a suitable place for storing any kind of secret.

Secrets should either be avoided (by using IAM roles) or should be a runtime concern and fetched with said IAM roles from a dedicated secrets manager like SSM parameter store, AWS SecretsManager, or something like OpenBao or Vault

12:16:19
@notmycommit:notwork.indbalanRedacted or Malformed Event14:41:19
@notmycommit:notwork.indbalanYup, We do this weird route more or less for avoiding storing secrets anywhere other than the instance.15:04:12
@notmycommit:notwork.indbalanSecrets are in vault or aws depending on the layer and they get populated on first boot in the config15:04:57
@adam:robins.wtf@adam:robins.wtfI wrote a simple module that will pull a secret down with the CLI given an ARN and some permissions. Creates a basic dir in /run to avoid storing them on disk22:51:20
@adam:robins.wtf@adam:robins.wtfAPI is roughly what you get from agenix22:51:47
@adam:robins.wtf@adam:robins.wtfWe do pull one secret during cloud-init, but otherwise try and keep it as simple as possible. Cloud init's main job is to discover the proper system store path, pull it, and switch to it, and a couple other imperative things about the system for PS1 and an env file. Our apps also now read their secrets directly on startup, so most secrets never get written outside memory.22:53:48
@kranzes:matrix.orgIlan Joselevich (Kranzes)
In reply to @adam:robins.wtf
I wrote a simple module that will pull a secret down with the CLI given an ARN and some permissions. Creates a basic dir in /run to avoid storing them on disk
Do you have this in a public repo somewhere?
23:04:17
29 Apr 2025
@adam:robins.wtf@adam:robins.wtfNo, it's all internal currently00:07:43
@adam:robins.wtf@adam:robins.wtfI'll try and remember to throw it in a gist at least01:30:26
@adam:robins.wtf@adam:robins.wtfIt's a very simplistic module :)01:30:37
@adam:robins.wtf@adam:robins.wtfhttps://gist.github.com/adamcstephens/9f45c33fa02270f669f2cd164faafea714:21:37
1 May 2025
@commiterate:matrix.orgcommiterate

might be easier to write a systemd unit (e.g. oneshot + optional timer or ExecStartPre) which does the credential pull than relying on boot user scripts.

Ideally the boot userscript just does a nixos-rebuild switch --flake {flake URL} and nothing else

19:46:13
@commiterate:matrix.orgcommiteratee.g. I'm using an ExecStartPre in a GitLab Runner systemd unit which fetches a GitLab Runner authn token from AWS Secrets Manager19:46:58

Show newer messages


Back to Room ListRoom Version: 10