| 14 Oct 2021 |
 K900 | So uh | 16:31:57 |
| K900 | I just checked in on it | 16:32:07 |
| K900 | And it crashed | 16:32:11 |
| K900 | With an allocator error | 16:32:14 |
| K900 | What even | 16:32:15 |
 Domen Kožar | K900: what is it? | 16:40:43 |
| K900 | My poor laptop trying to build rustc from source | 16:45:40 |
 Rick (Mindavi) | The allocator crashing with OOM sounds possible | 16:47:44 |
| K900 | It is not OOM | 16:48:26 |
| K900 | It's sigabrt'ing with something along the lines of "tried to realloc a pointer that's not allocated" | 16:48:44 |
| K900 | Oh my fucking god | 16:52:03 |
| K900 | I am a fucking idiot | 16:52:05 |
| Rick (Mindavi) | Finding all the bugs | 16:52:09 |
| K900 | I am building the wrong branch | 16:52:17 |
| K900 | Because I've got a deploy-rs dep that pulls in nixpkgs from nixos-unstable | 16:52:28 |
| K900 | And it's getting built with that | 16:52:34 |
| K900 | Fucking shoot me | 16:52:44 |
| K900 | lukegb (he/him): my sincerest apologies | 16:53:23 |
| K900 | I am a dumb | 16:53:25 |
 lukegb (he/him) | No worries, it happens :) | 17:37:52 |
| 18 Oct 2021 |
 @grahamc:nixos.org | I've got some time blocked off to bring up some capacity today | 13:16:10 |
|  Janne Heß joined the room. | 15:05:48 |
| @grahamc:nixos.org | I brought some mac capacity back online, and fixed the issues with deploying to the ephemeral aarch64 and x86 builders too. Hopefully this solves a lot of problems and we recover. Should I cancel queued, non-current builds? | 18:04:05 |
 tomberek | what were the problems and fixes? (mac2-guest and mac5guest seem to still be stuck) | 18:19:10 |
| @grahamc:nixos.org | I didn't get them all back online | 18:19:48 |
| lukegb (he/him) | Having some capacity back is an improvement though 👍 thanks so much! | 18:20:22 |
| @grahamc:nixos.org | one of the issues is fetching the installer at VM startup was periodically failing for ?reasons? and I moved it to fetch it at nixos-rebuild time, and provide the installer as part of the CONFIG ISO | 18:20:27 |
| lukegb (he/him) | We should get some more people trained up to fix up Hydra infra problems... | 18:20:45 |
| Janne Heß | In reply to @lukegb:zxcvbnm.ninja
We should get some more people trained up to fix up Hydra infra problems... If the need arises (and the trust is there of course), I could surely do some fixing. I've been operating a smaller-scale Hydra for a couple of years now together with ajs124 so I would say I do have some knowledge of Hydra and maintaining it | 18:46:48 |
| Janne Heß | In reply to @lukegb:zxcvbnm.ninja
We should get some more people trained up to fix up Hydra infra problems... * If the need arises (and the trust is there of course), I could surely do some fixing. I've been operating a smaller-scale Hydra for a couple of years now together with ajs124 so I would say I do have some knowledge of Hydra and maintaining it (although I'm aware that the general workflow differs greatly between how we are using Hydra and how nixos is using it) | 18:47:27 |
