!atvIbxHoEqNcAIxYpN:nixos.org

NixOS AWS

64 Members
16 Servers

Load older messages


SenderMessageTime
13 Apr 2025
@commiterate:matrix.orgcommiterateit's giving some motivation to set up testing + publishing automation for my rewrite02:32:02
@commiterate:matrix.orgcommiterateespecially now that I've figured out how to use EC2 Image Builder with Nix + system-manager or nix-darwin to do reproducible AMI baking (in this case it's just for testing the binary)02:32:49
@commiterate:matrix.orgcommiteratejust updated the repo to put the general bits in place, rest is just execution02:33:51
@arianvp:matrix.orgArianI just killed our last MacOS instance on ec209:58:34
@arianvp:matrix.orgArianDecided that dealing with MacOS in EC2 is not worth my time09:58:45
@arianvp:matrix.orgArianEspecially with how it interacts with nix (can't install nix on the ebs volume without clicking buttons in the UI)09:59:05
@commiterate:matrix.orgcommiterateit's pretty much unusable, though most of the blame is on Apple's EULA and TCC/FDA19:54:22
28 Apr 2025
@notmycommit:notwork.indbalanHey y'all not sure if this is documented somewhere, whats the reason AWS AMIs does not install aws cli by default? Is the expectation that we should bring it in with userdata?11:05:20
@arianvp:matrix.orgArian It’s not part of the amazon-image.nix profile. We could add it if you want? but yeh usually people just reconfigure their NixOS machines through user-data after boot 11:23:15
@arianvp:matrix.orgArianwhy do you need it?11:23:25
@notmycommit:notwork.indbalan Our scripts do some s3 gets / aws ssm get-parmeters etc to assemble the final nix config. We do have workaround for this to wrap the execution in nix-env. But IMO awscli seems quite useful on an amazon AMI. 11:31:24
@notmycommit:notwork.indbalan

Another option would be to look for nix-shell shebang, like

       #! /usr/bin/env nix-shell
       #! nix-shell -i runghc --packages 'haskellPackages.ghcWithPackages (ps: [ps.download-curl ps.tagsoup])'

in the init script where we determine the type in amazon-init.nix and run it with nix-shell instead of ${pkgs.runtimeShell}

11:51:29
@arianvp:matrix.orgArianwe download a closure and nixos-rebuild switch into ti11:57:01
@arianvp:matrix.orgArianwhy would you want to run imperative scripts if you can just deploy a new NixOS build with the script you want to run? Sounds more robust :D11:58:04
@notmycommit:notwork.indbalanHaha -- we use imperative secrets to assemble a nix config that we can switch to (mostly secrets and some other foo). But this is definitely not a blocker for us or anything.11:59:41
@arianvp:matrix.orgArian That sounds terrible :D 12:00:14
@arianvp:matrix.orgArian

you can also build your own ami. there are instructions in https://github.com/nixos/amis

But I highly advise against building NixOS configs with secrets in them. The nix store is world-readable and not a suitable place for storing any kind of secret.

Secrets should either be avoided (by using IAM roles) or should be a runtime concern and fetched with said IAM roles

12:14:52
@arianvp:matrix.orgAriananything else is bound to cause serious security issues12:15:20
@arianvp:matrix.orgArianAnd I can’t suggest int good faith to go that route12:15:33
@arianvp:matrix.orgArian * 12:15:41
@arianvp:matrix.orgArian *

you can also build your own ami. there are instructions in https://github.com/nixos/amis

But I highly advise against building NixOS configs with secrets in them. The nix store is world-readable and not a suitable place for storing any kind of secret.

Secrets should either be avoided (by using IAM roles) or should be a runtime concern and fetched with said IAM roles from a dedicated secrets manager like SSM parameter store, AWS SecretsManager, or something like OpenBao or Vault

12:16:19
@notmycommit:notwork.indbalanRedacted or Malformed Event14:41:19
@notmycommit:notwork.indbalanYup, We do this weird route more or less for avoiding storing secrets anywhere other than the instance.15:04:12
@notmycommit:notwork.indbalanSecrets are in vault or aws depending on the layer and they get populated on first boot in the config15:04:57
@adam:robins.wtfadamcstephensI wrote a simple module that will pull a secret down with the CLI given an ARN and some permissions. Creates a basic dir in /run to avoid storing them on disk22:51:20
@adam:robins.wtfadamcstephensAPI is roughly what you get from agenix22:51:47
@adam:robins.wtfadamcstephensWe do pull one secret during cloud-init, but otherwise try and keep it as simple as possible. Cloud init's main job is to discover the proper system store path, pull it, and switch to it, and a couple other imperative things about the system for PS1 and an env file. Our apps also now read their secrets directly on startup, so most secrets never get written outside memory.22:53:48
@kranzes:matrix.orgIlan Joselevich (Kranzes)
In reply to @adam:robins.wtf
I wrote a simple module that will pull a secret down with the CLI given an ARN and some permissions. Creates a basic dir in /run to avoid storing them on disk
Do you have this in a public repo somewhere?
23:04:17
29 Apr 2025
@adam:robins.wtfadamcstephensNo, it's all internal currently00:07:43
@adam:robins.wtfadamcstephensI'll try and remember to throw it in a gist at least01:30:26

Show newer messages


Back to Room ListRoom Version: 10