!CcTBuBritXGywOEGWJ:matrix.org

NixOS Binary Cache Self-Hosting

171 Members
About how to host a very large-scale binary cache and more58 Servers

Load older messages


SenderMessageTime
7 Mar 2024
@edef1c:matrix.orgedeflike, what i want from the incentive alignment is avoidance of "no, we won't pay you to spend time hacking on making this system more stable, but we will pay you to respond to the emergencies this creates"00:06:25
@adam:robins.wtfadamcstephens 🐝makes sense to me00:07:20
@whentze:matrix.orgWanja Hentze
In reply to @edef1c:matrix.org
part of the thing here is, you can prevent a lot of emergencies by doing consistent work to make them unlikely and handleable
yes. I'm currently on-call. The last time I was paged was months ago, and it was a false positive. the time before that was years ago.
00:07:30
@adam:robins.wtfadamcstephens 🐝that's the dream rotation00:07:45
@raitobezarius:matrix.orgraitobezarius
In reply to @whentze:matrix.org
yes. I'm currently on-call. The last time I was paged was months ago, and it was a false positive. the time before that was years ago.
do we need to make things spicier in nixos
00:07:51
@raitobezarius:matrix.orgraitobezariussorry00:07:53
@edef1c:matrix.orgedef
In reply to @whentze:matrix.org
yes. I'm currently on-call. The last time I was paged was months ago, and it was a false positive. the time before that was years ago.
and thus it would be a reasonable bet for me to take that you work in an environment where those incentives are well-aligned
00:08:01
@adam:robins.wtfadamcstephens 🐝i've definitely had on-call rotations that were much more painful than a couple times a year :)00:08:09
@whentze:matrix.orgWanja Hentzeif you do a good job on the infra and have reasonable requirements (this one is important), on-call can be relatively okay00:08:19
@whentze:matrix.orgWanja Hentzethat being said I would currently not join a 2nd rotation because I simply need time off for real once in a while00:11:49
@whentze:matrix.orgWanja HentzeI can help with infra work though if you want me to beat a NixOS over the head or replace a hard drive or whatever00:13:21
@linus:schreibt.jetzt@linus:schreibt.jetztYeah I'm not sure "on-call" makes that much sense here anyway? I'd consider it to be a best-effort thing, and I expect that if stuff catches fire people will scramble to fix it, but nobody needs to be paged out of whatever they're doing for it00:15:25
@whentze:matrix.orgWanja Hentzeidk, crates.io has a rotation and I consider that less essential00:16:07
@linus:schreibt.jetzt@linus:schreibt.jetzt we also have the advantage of the cache not being super critical for nix/nixos to be usable 00:16:16
@linus:schreibt.jetzt@linus:schreibt.jetztlike if it goes down for a day, that would be much less of a problem than the ubuntu package servers being down for a day would be for ubuntu00:17:10
@linus:schreibt.jetzt@linus:schreibt.jetztergh, I should sleep, not think about this, for now00:19:46
@linus:schreibt.jetzt@linus:schreibt.jetztgood night :D00:19:48
@whentze:matrix.orgWanja Hentzeeh, for a loooot of NixOS users, the cache being down means nixos-rebuild just becomes a desk warming app00:19:53
@edef1c:matrix.orgedefyeah00:20:03
@edef1c:matrix.orgedeffrom the cache logs, i know we have a lot of users in like, low-GDP countries you've never heard of before00:20:30
@whentze:matrix.orgWanja Hentzewe can actually build our closures on our CI infra but I'm sure many can't00:20:36
@edef1c:matrix.orgedefi don't think they would appreciate the desk warming since it is coming out of their power bill00:20:46
@whentze:matrix.orgWanja Hentzeand in my hobby usage I definitely can't00:20:55
@edef1c:matrix.orgedefnor do they likely have the RAM to actually build most things00:21:11
@raitobezarius:matrix.orgraitobezarius
In reply to @edef1c:matrix.org
from the cache logs, i know we have a lot of users in like, low-GDP countries you've never heard of before
i wonder if low GDP countries can download fast enough some of our stuff though
00:21:31
@raitobezarius:matrix.orgraitobezarius and if they don't end up passing --no-substituters or something 00:21:42
@edef1c:matrix.orgedefi'm working from cache logs, so they do seem to hit it00:22:01
@edef1c:matrix.orgedefor they wouldn't show on my radar at all00:22:05
@raitobezarius:matrix.orgraitobezariusah right00:22:05
@edef1c:matrix.orgedefbandwidth will hopefully improve in the long run if we can get deduping to work well and carry all the way through to the frontend protocol00:22:55

Show newer messages


Back to Room ListRoom Version: 10