!atvIbxHoEqNcAIxYpN:nixos.org

NixOS AWS

61 Members
15 Servers

Load older messages


SenderMessageTime
4 May 2025
@arianvp:matrix.orgArianWe put the nix store path in the user-data at work and then reboot into the new config09:05:08
@arianvp:matrix.orgArianso our hosts only need to have access to S309:05:24
@arianvp:matrix.orgArianto substitute09:05:29
@arianvp:matrix.orgArian though the S3 substituter in Nix is completely broken beyond belief 09:05:37
@arianvp:matrix.orgArianI need to find time to rewrite it09:05:40
@arianvp:matrix.orgArianhttps://github.com/NixOS/nix/issues/1308409:05:54
@arianvp:matrix.orgArian * 09:06:13
@p14:matrix.orgp14 Arian: oof! That sounds pretty nasty, though the fix is looking quite simple. How far out is the fix? It sounds like you've done a lot of the work for it already... 11:43:47
5 May 2025
@arianvp:matrix.orgArian

Hey all, I have a message from the foundation I think makes sense to forward here:

⁦Hey everyone, happy almost‑Monday for some, and full‑on Monday for most! 😉

We’re kicking off our AWS cache sponsorship renewal (shoutout to the AWS Open Source teams for sponsoring us for two consecutive years now) and could use your help. If you work at AWS, or know someone who does, please drop me a note!

Each year we put together a summary of how Nix adds value on AWS, and real‑world examples make our case much stronger. If you’ve used Nix for anything on EC2, Lambda, EKS, CodeBuild, caching Docker layers, reducing build times, improving deploy consistency, or anything else in the AWS ecosystem, I’d love to hear about it. Even a quick bullet point or link to a project helps!

Thanks in advance for any pointers or introductions you can share. 🙏

07:48:56
@arianvp:matrix.orgArian So e.g. @commiterate:matrix.orgor @adam:robins.wtfif you are in a position where you could share a blurb about how you're using AWS + Nix at your company that'd be really awesome. No obligation of course. 07:51:21
6 May 2025
@commiterate:matrix.orgcommiterate

We use Nix + system-manager to bake reproducible Amazon Linux 2023 AMIs. There's a shell script snippet in this GitHub issue: https://github.com/aws/ec2-image-builder-roadmap/issues/110

# Switch from ssm-user to the default user.
sudo su ec2-user

# Install RPM packages.
sudo dnf install --assumeyes curl-minimal git

# Install Nix.
curl --fail --location https://install.determinate.systems/nix/tag/v3.1.1 --proto '=https' --show-error --silent --tlsv1.2 | sh -s -- install --no-confirm
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh

# Setup Nix flake registry.
nix registry add nixpkgs github:NixOS/nixpkgs/{Git revision hash}
nix registry add system-manager github:numtide/system-manager/{Git revision hash}

# Install Nix packages.
nix profile install system-manager

# Apply system-manager configuration (installs system-wide packages and sets up systemd units).
sudo $(command -v system-manager) pre-populate --flake 'git+{Git HTTPs URL}&rev={Git revision hash}#{system-manager flake output key}'

CloudFormation templates are generated with the AWS CDK. The infrastructure code essentially:

  • Locks an Amazon-managed AMI ARN from their SSM public parameters into the cdk.context.json file.
  • Calls nix flake metadata to get the flake's Git hash and construct a flake URL (we run Nix eval on a system in a VPC with company network connectivity).
  • Generates an SSM document with the flake URL.
  • Sets up the EC2 Image Builder infrastructure that auto-builds an AMI on CloudFormation stack deploys.
  • Reference the AMI in a launch template which is then used in an auto-scaling group.
  • Use CloudFormation rolling update to bounce the auto-scaling group (until ASG instance refresh is supported in CloudFormation).

Once the CloudFormation image import situation is improved, we'll move the non-bootstrap stuff to use NixOS disk images created with the systemd-repart helpers.

18:10:47
@commiterate:matrix.orgcommiterate *

We use Nix + system-manager to bake reproducible Amazon Linux 2023 AMIs. There's a shell script snippet in this GitHub issue: https://github.com/aws/ec2-image-builder-roadmap/issues/110

# Switch from ssm-user to the default user.
sudo su ec2-user

# Install RPM packages.
sudo dnf install --assumeyes curl-minimal git

# Install Nix.
curl --fail --location https://install.determinate.systems/nix/tag/v3.1.1 --proto '=https' --show-error --silent --tlsv1.2 | sh -s -- install --no-confirm
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh

# Setup Nix flake registry.
nix registry add nixpkgs github:NixOS/nixpkgs/{Git revision hash}
nix registry add system-manager github:numtide/system-manager/{Git revision hash}

# Install Nix packages.
nix profile install system-manager

# Apply system-manager configuration (installs system-wide packages and sets up systemd units).
sudo $(command -v system-manager) pre-populate --flake 'git+{Git HTTPs URL}&rev={Git revision hash}#{system-manager flake output key}'

CloudFormation templates are generated with the AWS CDK. The infrastructure code essentially:

  • Locks an Amazon-managed AMI ARN from their SSM public parameters into the cdk.context.json file.
  • Calls nix flake metadata to get the flake's Git hash and construct a flake URL (we run Nix eval on a system in a VPC with company network connectivity).
  • Generates an SSM document with the flake URL.
  • Sets up the EC2 Image Builder infrastructure that auto-builds an AMI on CloudFormation stack deploys.
  • Reference the AMI in a launch template which is then used in an auto-scaling group.
  • Use CloudFormation rolling update to bounce the auto-scaling group (until ASG instance refresh is supported in CloudFormation).

Once the CloudFormation image import situation is improved, we'll move the non-bootstrap stuff to use NixOS disk images created with the systemd-repart helpers.

18:11:04
@commiterate:matrix.orgcommiterate *

We use Nix + system-manager to bake reproducible Amazon Linux 2023 AMIs. There's a shell script snippet in this GitHub issue: https://github.com/aws/ec2-image-builder-roadmap/issues/110

# Switch from ssm-user to the default user.
sudo su ec2-user

# Install RPM packages.
sudo dnf install --assumeyes curl-minimal git

# Install Nix.
curl --fail --location https://install.determinate.systems/nix/tag/v3.1.1 --proto '=https' --show-error --silent --tlsv1.2 | sh -s -- install --no-confirm
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh

# Setup Nix flake registry.
nix registry add nixpkgs github:NixOS/nixpkgs/{Git revision hash}
nix registry add system-manager github:numtide/system-manager/{Git revision hash}

# Install Nix packages.
nix profile install system-manager

# Apply system-manager configuration (installs system-wide packages and sets up systemd units).
sudo $(command -v system-manager) pre-populate --flake 'git+{Git HTTPs URL}&rev={Git revision hash}#{system-manager flake output key}'

CloudFormation templates are generated with the AWS CDK. The infrastructure code essentially:

  • Locks an Amazon-managed AMI ARN from their SSM public parameters into the cdk.context.json file.
  • Calls nix flake metadata to get the flake's Git hash and construct a flake URL for the flake itself and some inputs needed in the script (we run Nix eval on a system in a VPC with company network connectivity).
  • Generates an SSM document with the flake URL.
  • Sets up the EC2 Image Builder infrastructure that auto-builds an AMI on CloudFormation stack deploys.
  • Reference the AMI in a launch template which is then used in an auto-scaling group.
  • Use CloudFormation rolling update to bounce the auto-scaling group (until ASG instance refresh is supported in CloudFormation).

Once the CloudFormation image import situation is improved, we'll move the non-bootstrap stuff to use NixOS disk images created with the systemd-repart helpers.

18:12:11
@commiterate:matrix.orgcommiterate *

We use Nix + system-manager to bake reproducible Amazon Linux 2023 AMIs. There's a shell script snippet in this GitHub issue: https://github.com/aws/ec2-image-builder-roadmap/issues/110

# Switch from ssm-user to the default user.
sudo su ec2-user

# Install RPM packages.
sudo dnf install --assumeyes curl-minimal git

# Install Nix.
curl --fail --location https://install.determinate.systems/nix/tag/v3.1.1 --proto '=https' --show-error --silent --tlsv1.2 | sh -s -- install --no-confirm
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh

# Setup Nix flake registry.
nix registry add nixpkgs github:NixOS/nixpkgs/{Git revision hash}
nix registry add system-manager github:numtide/system-manager/{Git revision hash}

# Install Nix packages.
nix profile install system-manager

# Apply system-manager configuration (installs system-wide packages and sets up systemd units).
sudo $(command -v system-manager) pre-populate --flake 'git+{Git HTTPs URL}&rev={Git revision hash}#{system-manager flake output key}'

CloudFormation templates are generated with the AWS CDK. The infrastructure code essentially:

  • Locks an Amazon-managed AMI ARN from their SSM public parameters into the cdk.context.json file.
  • Calls nix flake metadata to get the flake's Git hash and construct a flake URL for the flake itself and some inputs needed in the script (we run Nix eval on a system in a VPC with company network connectivity).
  • Generates an SSM document with the flake URL.
  • Sets up the EC2 Image Builder infrastructure that auto-builds an AMI on CloudFormation stack deploys.
  • Reference the AMI in a launch template which is then used in an auto-scaling group.
  • Use CloudFormation rolling update to bounce the auto-scaling group (until ASG instance refresh is supported in CloudFormation).

Once the CloudFormation image import situation is improved, we'll move the non-bootstrap stuff to use NixOS disk images created with the systemd-repart helpers.

This ends up being fully reproducible because Amazon Linux 2023 locks the Amazon Linux package repository version (these are globally versioned now), so any Nix dependencies like curl and git locked by the AMI itself.

18:13:00
@commiterate:matrix.orgcommiterate *

We use Nix + system-manager to bake reproducible Amazon Linux 2023 AMIs. There's a shell script snippet in this GitHub issue: https://github.com/aws/ec2-image-builder-roadmap/issues/110

# Switch from ssm-user to the default user.
sudo su ec2-user

# Install RPM packages.
sudo dnf install --assumeyes curl-minimal git

# Install Nix.
curl --fail --location https://install.determinate.systems/nix/tag/v3.1.1 --proto '=https' --show-error --silent --tlsv1.2 | sh -s -- install --no-confirm
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh

# Setup Nix flake registry.
nix registry add nixpkgs github:NixOS/nixpkgs/{Git revision hash}
nix registry add system-manager github:numtide/system-manager/{Git revision hash}

# Install Nix packages.
nix profile install system-manager

# Apply system-manager configuration (installs system-wide packages and sets up systemd units).
sudo $(command -v system-manager) pre-populate --flake 'git+{Git HTTPs URL}&rev={Git revision hash}#{system-manager flake output key}'

CloudFormation templates are generated with the AWS CDK. The infrastructure code essentially:

  • Locks an Amazon-managed AMI ARN from their SSM public parameters into the cdk.context.json file.
  • Calls nix flake metadata to get the flake's Git hash and construct a flake URL for the flake itself and some inputs needed in the script (we run Nix eval on a system in a VPC with company network connectivity).
  • Generates an SSM document with the flake URL.
  • Sets up the EC2 Image Builder infrastructure that auto-builds an AMI on CloudFormation stack deploys.
  • Reference the AMI in a launch template which is then used in an auto-scaling group.
  • Use CloudFormation rolling update to bounce the auto-scaling group (until ASG instance refresh is supported in CloudFormation).

Once the CloudFormation image import situation is improved, we'll move the non-bootstrap stuff to use NixOS disk images created with the systemd-repart helpers.

This ends up being fully reproducible because Amazon Linux 2023 locks the Amazon Linux package repository version (these are globally versioned now), so any Nix dependencies like curl and git locked by the base Amazon Linux 2023 AMI itself.

18:13:09
@commiterate:matrix.orgcommiterate *

We use Nix + system-manager to bake reproducible Amazon Linux 2023 AMIs. There's a shell script snippet in this GitHub issue: https://github.com/aws/ec2-image-builder-roadmap/issues/110

# Switch from ssm-user to the default user.
sudo su ec2-user

# Install RPM packages.
sudo dnf install --assumeyes curl-minimal git

# Install Nix.
curl --fail --location https://install.determinate.systems/nix/tag/v3.1.1 --proto '=https' --show-error --silent --tlsv1.2 | sh -s -- install --no-confirm
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh

# Setup Nix flake registry.
nix registry add nixpkgs github:NixOS/nixpkgs/{Git revision hash}
nix registry add system-manager github:numtide/system-manager/{Git revision hash}

# Install Nix packages.
nix profile install system-manager

# Apply system-manager configuration (installs system-wide packages and sets up systemd units).
sudo $(command -v system-manager) pre-populate --flake 'git+{Git HTTPs URL}&rev={Git revision hash}#{system-manager flake output key}'

CloudFormation templates are generated with the AWS CDK. The infrastructure code essentially:

  • Locks an Amazon-managed AMI ARN from their SSM public parameters into the cdk.context.json file.
  • Calls nix flake metadata to get the flake's Git hash and construct a flake URL for the flake itself and some inputs needed in the script (we run Nix eval on a system in a VPC with company network connectivity).
  • Generates an SSM document with the flake URL.
  • Sets up the EC2 Image Builder infrastructure that auto-builds an AMI on CloudFormation stack deploys.
  • Reference the AMI in a launch template which is then used in an auto-scaling group.
  • Use CloudFormation rolling update to bounce the auto-scaling group (until ASG instance refresh is supported in CloudFormation).

Once the CloudFormation image import situation is improved, we'll move the non-bootstrap stuff to use NixOS disk images created with the systemd-repart helpers.

This ends up being fully reproducible because Amazon Linux 2023 locks the Amazon Linux package repository version (these are globally versioned now), so any Nix dependencies like curl and git are locked by the base Amazon Linux 2023 AMI itself.

18:19:33
@arianvp:matrix.orgArianCan you disclose what company you work for? If not also no issue. I18:26:47
@arianvp:matrix.orgArian* Can you disclose what company you work for? If not also no issue. 18:27:04
@commiterate:matrix.orgcommiterateNVIDIA (though I'm on the DGX Cloud side, not the GPU side. There's some internal Nix users pushing better NixOS NVIDIA driver support though)18:27:35
@commiterate:matrix.orgcommiterate* NVIDIA (though I'm on the DGX Cloud side, not the GPU side. There's some internal Nix users pushing for better NixOS NVIDIA driver support though)18:27:48
@commiterate:matrix.orgcommiterate *

We use Nix + system-manager to bake reproducible Amazon Linux 2023 AMIs. There's a shell script snippet in this GitHub issue: https://github.com/aws/ec2-image-builder-roadmap/issues/110

# Switch from ssm-user to the default user.
sudo su ec2-user

# Install RPM packages.
sudo dnf install --assumeyes curl-minimal git

# Install Nix.
curl --fail --location https://install.determinate.systems/nix/tag/v3.1.1 --proto '=https' --show-error --silent --tlsv1.2 | sh -s -- install --no-confirm
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh

# Setup Nix flake registry.
nix registry add nixpkgs github:NixOS/nixpkgs/{Git revision hash}
nix registry add system-manager github:numtide/system-manager/{Git revision hash}

# Install Nix packages.
nix profile install system-manager

# Apply system-manager configuration (installs system-wide packages and sets up systemd units).
sudo $(command -v system-manager) pre-populate --flake 'git+{Git HTTPs URL}&rev={Git revision hash}#{system-manager flake output key}'

CloudFormation templates are generated with the AWS CDK. The infrastructure code essentially:

  • Locks an Amazon-managed AMI ARN from their SSM public parameters into the cdk.context.json file.
  • Calls nix flake metadata to get the flake's Git hash and construct a flake URL for the flake itself and some inputs needed in the script (we run Nix eval on a system in a VPC with company network connectivity).
  • Generates an SSM document with the aforementioned script.
  • Sets up the EC2 Image Builder infrastructure that auto-builds an AMI on CloudFormation stack deploys.
  • Reference the AMI in a launch template which is then used in an auto-scaling group.
  • Use CloudFormation rolling update to bounce the auto-scaling group (until ASG instance refresh is supported in CloudFormation).

Once the CloudFormation image import situation is improved, we'll move the non-bootstrap stuff to use NixOS disk images created with the systemd-repart helpers.

This ends up being fully reproducible because Amazon Linux 2023 locks the Amazon Linux package repository version (these are globally versioned now), so any Nix dependencies like curl and git are locked by the base Amazon Linux 2023 AMI itself.

18:38:52
@commiterate:matrix.orgcommiterate *

We use Nix + system-manager to bake reproducible Amazon Linux 2023 AMIs. There's a shell script snippet in this GitHub issue: https://github.com/aws/ec2-image-builder-roadmap/issues/110

# Switch from ssm-user to the default user.
sudo su ec2-user

# Install RPM packages.
sudo dnf install --assumeyes curl-minimal git

# Install Nix.
curl --fail --location https://install.determinate.systems/nix/tag/v3.1.1 --proto '=https' --show-error --silent --tlsv1.2 | sh -s -- install --no-confirm
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh

# Setup Nix flake registry.
nix registry add nixpkgs github:NixOS/nixpkgs/{Git revision hash}
nix registry add system-manager github:numtide/system-manager/{Git revision hash}

# Install Nix packages.
nix profile install system-manager

# Apply system-manager configuration (installs system-wide packages and sets up systemd units).
sudo $(command -v system-manager) pre-populate --flake 'git+{Git HTTPs URL}&rev={Git revision hash}#{system-manager flake output key}'

CloudFormation templates are generated with the AWS CDK. The infrastructure code essentially:

  • Locks an Amazon-managed AMI ID from their SSM public parameters into the cdk.context.json file.
  • Calls nix flake metadata to get the flake's Git hash and construct a flake URL for the flake itself and some inputs needed in the script (we run Nix eval on a system in a VPC with company network connectivity).
  • Generates an SSM document with the aforementioned script.
  • Sets up the EC2 Image Builder infrastructure that auto-builds an AMI on CloudFormation stack deploys.
  • Reference the AMI in a launch template which is then used in an auto-scaling group.
  • Use CloudFormation rolling update to bounce the auto-scaling group (until ASG instance refresh is supported in CloudFormation).

Once the CloudFormation image import situation is improved, we'll move the non-bootstrap stuff to use NixOS disk images created with the systemd-repart helpers.

This ends up being fully reproducible because Amazon Linux 2023 locks the Amazon Linux package repository version (these are globally versioned now), so any Nix dependencies like curl and git are locked by the base Amazon Linux 2023 AMI itself.

18:49:07
@commiterate:matrix.orgcommiterate *

We use Nix + system-manager to bake reproducible Amazon Linux 2023 AMIs. There's a shell script snippet in this GitHub issue: https://github.com/aws/ec2-image-builder-roadmap/issues/110

# Switch from ssm-user to the default user.
sudo su ec2-user

# Install RPM packages.
sudo dnf install --assumeyes curl-minimal git

# Install Nix.
curl --fail --location https://install.determinate.systems/nix/tag/v3.1.1 --proto '=https' --show-error --silent --tlsv1.2 | sh -s -- install --no-confirm
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh

# Setup Nix flake registry.
nix registry add nixpkgs github:NixOS/nixpkgs/{Git revision hash}
nix registry add system-manager github:numtide/system-manager/{Git revision hash}

# Install Nix packages.
nix profile install system-manager

# Apply system-manager configuration (installs system-wide packages and sets up systemd units).
sudo $(command -v system-manager) pre-populate --flake 'git+{Git HTTPs URL}&rev={Git revision hash}#{system-manager flake output key}'

CloudFormation templates are generated with the AWS CDK. The infrastructure code essentially:

  • Locks an Amazon-managed AMI ID from their SSM public parameters into the cdk.context.json file.
  • Calls nix flake metadata to get the flake's Git hash and construct a flake URL for the flake itself and some inputs needed in the script (we run Nix eval on a system in a VPC with company network connectivity).
  • Generates an SSM document with the aforementioned script.
  • Sets up the EC2 Image Builder infrastructure that auto-builds an AMI on CloudFormation stack deploys.
  • Reference the AMI in a launch template which is then used in an auto-scaling group.
  • Use CloudFormation rolling update to bounce the auto-scaling group (until ASG instance refresh is supported in CloudFormation).

Once the CloudFormation EC2 disk image import situation is improved, we'll move the non-bootstrap stuff to use NixOS disk images created with the systemd-repart helpers.

This ends up being fully reproducible because Amazon Linux 2023 locks the Amazon Linux package repository version (these are globally versioned now), so any Nix dependencies like curl and git are locked by the base Amazon Linux 2023 AMI itself.

18:55:50
@adam:robins.wtf@adam:robins.wtfWe recently migrated to NixOS for our application servers, and have a couple more instance profiles before we'll be fully on NixOS. We're deploying Elixir applications with an in-house deployment tool, leveraging S3 for deployment coordination and as a binary cache. We build Nix paths for NixOS and applications separately, push them to the cache and write those paths to a bucket along with other metadata including the git sha. Then during deploy we pull and activate those paths (no eval). While we do build our own AMI, we also use this same path-based deployment through amazon-init to switch to the correct profile on boot.18:56:00
@adam:robins.wtf@adam:robins.wtfI work for a company named CalmWave. We're a US-based healthcare startup.18:56:36
7 May 2025
@randoneering:matrix.orgjustin@randoneering joined the room.00:35:19
15 May 2025
@commiterate:matrix.orgcommiterate* NVIDIA (though I'm on the DGX Cloud side, not the GPU side. There's some internal Nix users pushing for better NixOS NVIDIA driver + CUDA support though)07:33:09
18 May 2025
@arianvp:matrix.orgArian

urgh:

https://github.com/aws/amazon-ec2-metadata-mock/issues/234

12:14:29
@commiterate:matrix.orgcommiterateTurnaround on most of the AWS open source stuff outside of the AWS CDK or SDKs (which have dedicated support engineers) is pretty terrible.17:26:05
@commiterate:matrix.orgcommiterate* Turnaround on most of the AWS open source stuff outside of the AWS CDK or CLI + SDKs (which have dedicated support engineers) is pretty terrible.17:26:16

Show newer messages


Back to Room ListRoom Version: 10