| 19 Oct 2024 |
 hexa (signing key rotation when) | one last thing: the arm builder at hetzner works with nix 2.18.8, while the one at EM with nix 2.24.9 does not | 22:55:20 |
 vcunat | I also give up for today. | 22:59:38 |
| hexa (signing key rotation when) | tomorrow we might try to rollback hydra to the working 2.19 state on 24.05 | 23:00:47 |
| hexa (signing key rotation when) | maybe that'll make things work, who knows | 23:00:59 |
| 20 Oct 2024 |
| hexa (signing key rotation when) | ok, apparently I'm still awake | 00:14:50 |
| hexa (signing key rotation when) | and puck came up with a fix | 00:15:06 |
| hexa (signing key rotation when) | the EM hosts were the only ones affected, because they specify an ssh host key in their machine definition | 00:15:29 |
| hexa (signing key rotation when) | that gets decoded once in hydra, then passed to nix, which tries to decode it again … which fails | 00:15:49 |
| hexa (signing key rotation when) | so we are back in production | 00:16:07 |
| hexa (signing key rotation when) | the builders are currently running on 2.18.8, but I reverted back to unpinning the nix version again | 00:16:36 |
| hexa (signing key rotation when) | so they'll slowly migrate to 2.24 over the next few days | 00:16:46 |
 Mic92 | hexa (signing key rotation when): thanks. I think John Ericson added some better error handling for parsing machine definitions to nix, because I think he faced a similar issue? Can you give me the failing machine definition? | 11:19:25 |
| hexa (signing key rotation when) | e.g. root@2d838f2f.packethost.net x86_64-linux,i686-linux /var/lib/hydra/queue-runner/.ssh/id_ed25519 24 1 kvm,nixos-test - c3NoLWVkMjU1MTkgQUFBQUMzTnphQzFsWkRJMU5URTVBQUFBSU8yWXpEZzdHZ3Y5K3lDNVhCeWlNcW1wbE5WTFhwZFdKcUhTc0tRYXpJU0Q= | 12:32:12 |
| Mic92 | We were thinking about setting up a staging hydra to test the upcoming s3 cleanup (my plan b). I also want to use it to test patches that you were pointing out. | 12:56:04 |
 John Ericson | Yeah maybe what happened was when I started using Nix's machine file parsing things got decoded twice | 14:13:30 |
| John Ericson | Ah I see you all found it already, good! | 14:44:02 |
| John Ericson | Let's get that fix on the `nix-2.21` branch and then merge it into each successive `nix-*` branch and master. | 14:46:16 |
| Mic92 | John Ericson: I would like to document this branching model that you use for hydra. How does it work? And what nix branches do we actually want to support? | 16:25:57 |
 emily | given that Nix only support 2.24 and (for a few more months) 2.18 it seems pointless to go further for Hydra | 16:27:08 |
| John Ericson | @joerg:thalheim.io: (terse cause phone) basically instead of back ports I do merges | 16:27:19 |
| John Ericson | You can re-merge older nix-* into new nix-* all you want | 16:27:54 |
| John Ericson | master uses the latest nix release | 16:28:09 |
| John Ericson | nix-next uses Nix master | 16:28:18 |
| John Ericson | nix-* has "the newest stuff" as long as it uses the right nix version | 16:29:43 |
| John Ericson | So sometimes it's convenient to make a change in an older nix-* and then merge merge merge until on reaches master | 16:30:45 |
| hexa (signing key rotation when) | Mic92: how does the devshell work in the terraform dir? | 23:22:09 |
| hexa (signing key rotation when) | I get neither aws nor terraform even though I have opted into direnv | 23:22:20 |
| hexa (signing key rotation when) | hm ok, it seemingly got confused by the nested devenvs | 23:29:19 |
| 21 Oct 2024 |
 nh2 | WHo manages the AWS AMIs?
This one is missing already for 6 months with no reply:
https://github.com/NixOS/nixpkgs/issues/307519 | 02:10:28 |
| emily | Arian might know | 02:11:24 |
