| 7 Mar 2024 |
edef | the people you retain are the ones for whom the incentives work | 00:02:32 |
edef | this scene has bled people pretty hard over the ~8y i've been part of the show | 00:03:10 |
@adam:robins.wtf | I do think we should expect to pay someone for an ongoing number of hours. While it may not be necessary to care and feed a system every day, I wouldn't underestimate the potential problems or just unforeseen maintenance of a storage cluster. | 00:03:13 |
edef | part of the thing here is, you can prevent a lot of emergencies by doing consistent work to make them unlikely and handleable | 00:04:27 |
edef | spes non consilium est; hope is not a strategy | 00:04:49 |
edef | consistently building reliable systems and doing reliability work instead of responding to incidents is how we make this good | 00:05:19 |
edef | and this buys us various kinds of budget to do the improving, it compounds well | 00:05:40 |
edef | like, what i want from the incentive alignment is avoidance of "no, we won't pay you to spend time hacking on making this system more stable, but we will pay you to respond to the emergencies this creates" | 00:06:25 |
@adam:robins.wtf | makes sense to me | 00:07:20 |
Wanja Hentze | In reply to @edef1c:matrix.org part of the thing here is, you can prevent a lot of emergencies by doing consistent work to make them unlikely and handleable yes. I'm currently on-call. The last time I was paged was months ago, and it was a false positive. the time before that was years ago. | 00:07:30 |
@adam:robins.wtf | that's the dream rotation | 00:07:45 |
raitobezarius | In reply to @whentze:matrix.org yes. I'm currently on-call. The last time I was paged was months ago, and it was a false positive. the time before that was years ago. do we need to make things spicier in nixos | 00:07:51 |
raitobezarius | sorry | 00:07:53 |
edef | In reply to @whentze:matrix.org yes. I'm currently on-call. The last time I was paged was months ago, and it was a false positive. the time before that was years ago. and thus it would be a reasonable bet for me to take that you work in an environment where those incentives are well-aligned | 00:08:01 |
@adam:robins.wtf | i've definitely had on-call rotations that were much more painful than a couple times a year :) | 00:08:09 |
Wanja Hentze | if you do a good job on the infra and have reasonable requirements (this one is important), on-call can be relatively okay | 00:08:19 |
Wanja Hentze | that being said I would currently not join a 2nd rotation because I simply need time off for real once in a while | 00:11:49 |
Wanja Hentze | I can help with infra work though if you want me to beat a NixOS over the head or replace a hard drive or whatever | 00:13:21 |
@linus:schreibt.jetzt | Yeah I'm not sure "on-call" makes that much sense here anyway? I'd consider it to be a best-effort thing, and I expect that if stuff catches fire people will scramble to fix it, but nobody needs to be paged out of whatever they're doing for it | 00:15:25 |
Wanja Hentze | idk, crates.io has a rotation and I consider that less essential | 00:16:07 |
@linus:schreibt.jetzt | we also have the advantage of the cache not being super critical for nix/nixos to be usable | 00:16:16 |
@linus:schreibt.jetzt | like if it goes down for a day, that would be much less of a problem than the ubuntu package servers being down for a day would be for ubuntu | 00:17:10 |
@linus:schreibt.jetzt | ergh, I should sleep, not think about this, for now | 00:19:46 |
@linus:schreibt.jetzt | good night :D | 00:19:48 |
Wanja Hentze | eh, for a loooot of NixOS users, the cache being down means nixos-rebuild just becomes a desk warming app | 00:19:53 |
edef | yeah | 00:20:03 |
edef | from the cache logs, i know we have a lot of users in like, low-GDP countries you've never heard of before | 00:20:30 |
Wanja Hentze | we can actually build our closures on our CI infra but I'm sure many can't | 00:20:36 |
edef | i don't think they would appreciate the desk warming since it is coming out of their power bill | 00:20:46 |
Wanja Hentze | and in my hobby usage I definitely can't | 00:20:55 |