| 27 Feb 2025 |
 drewhaven | Probably the main thing is that we already have the access controls set up. | 23:27:27 |
| drewhaven | A former SRE was going to use it for this, but never got around to it. Now I'm taking a crack at it. | 23:28:02 |
| drewhaven | I'm evaluating options atm. | 23:28:10 |
| drewhaven set a profile picture. | 23:29:15 |
| 28 Feb 2025 |
 adamcstephens | ssh is superior to ssm, except for the fact that ssm can use sso through aws. | 00:07:22 |
| adamcstephens | not that you can't get sso with regular linux, but the setup ease of ssm is probably hard to beat. | 00:08:25 |
| adamcstephens | SSH/SSM/SSO, are there other such acronyms? | 00:09:07 |
| drewhaven | Don't forget SSL | 00:14:32 |
 commiterate | SSM's main benefit is not needing to open up any security group or VPC rules to allow SSH ingress. I'd only use it for quick debugging though. | 00:29:32 |
| commiterate | The console terminal gets a bit annoying to use for anything more complicated. | 00:30:01 |
| commiterate | * The browser WebSockets terminal gets a bit annoying to use for anything more complicated. | 00:30:14 |
| drewhaven | That's the main use-case here, being able to run some commands on the deployed fleet and occasionally connect if something seems off. Our current system has an awkward setup where each box has it's own OpenVPN connection to a central EC2 instance. | 00:34:05 |
| drewhaven | The local network is unmanaged and untrusted. | 00:34:26 |
| drewhaven | * The local network for the deployed boxes is unmanaged and untrusted. | 00:34:34 |
| commiterate | yup SSM seems perfect then. It's significantly easier than trying to stand up your own bastion hosts | 00:44:05 |
| commiterate | * yup SSM seems perfect then. It's significantly easier than trying to stand up your own bastion/ops hosts | 00:45:30 |
| commiterate | you don't have to use the system update features of SSM so it can work with immutable stuff | 00:48:58 |
| commiterate | that's basically how AWS use it themselves internally | 00:49:32 |
| commiterate | bake an immutable Amazon Linux AMI with the SSM agent then only use it for the occasional deeper debug (assuming CW metrics + logs don't have enough info) | 00:50:07 |
| commiterate | * bake an immutable Amazon Linux AMI with the SSM agent then only use it for the occasional deeper debug (when the CW metrics + logs don't have enough info) | 00:52:39 |
| drewhaven | Exactly. The goal is to make the system so simple that the vast majority of issues can be fixed with a reboot or rollback. | 01:02:19 |
| drewhaven | Now to keep designing the rest of the system and make sure everyone's onboard with NixOS. Not everyone's as comfortable with Nix as I am, but it's a huge step up from the years-out-of-date Fedora they were using where every system could be a bit different. | 01:03:31 |
| commiterate | If you're using Cfn to manage your immutable infra setup, ASG instance refresh in Cfn stack updates is planned for this year and I'm chasing them to allow Cfn-managed S3 -> EBS snapshot imports. Latter is still waiting for a meeting with a PM though. | 01:05:43 |
| commiterate | Once both of those are in place, AWS CDK-based apps should be able to build a raw NixOS disk image wtih Nix, mark it as a CDK asset, and reference it in the AMI import resource. | 01:06:37 |
| commiterate | * If you're using Cfn to manage your immutable infra setup, ASG instance refresh in Cfn stack updates is planned for this year and I'm chasing them to allow Cfn-managed S3 -> EBS snapshot imports. Latter is still waiting for a meeting with a PM though.
- https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/2119
- https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/2175
| 01:07:42 |
| commiterate | * Once both of those are in place, AWS CDK-based apps should be able to build a raw NixOS disk image with Nix, mark it as a CDK asset, and reference it in the AMI import resource. | 04:17:49 |
 Arian | Wild that they dont have instance refresh | 07:28:33 |
| Arian | In all fairness. Terraform's implementation is bugged. So we trigger instance refresh with the CLI too | 07:28:52 |
| commiterate | I'm surprised Cfn support wasn't a launch requirement since that's basically required for most AWS services. | 17:11:39 |
| commiterate | Though instance refresh is a bit different since it's an action rather than a resource. | 17:12:01 |
