!CcTBuBritXGywOEGWJ:matrix.org

NixOS Binary Cache Self-Hosting

160 Members
About how to host a very large-scale binary cache and more55 Servers

You have reached the beginning of time (for this room).


SenderMessageTime
6 Mar 2024
@edef1c:matrix.orgedeflike, broadly the cache is in a weird spot because a) it's infrastructure. nobody gives a shit if you do your job right, and nothing has been deleted or noticeably broken yet b) it has been neglected for ages, this project should have started a years ago, not six months ago, and it shouldn't have depended on me as this much of a SPOF c) it's a public good. there is no value capture story for it, if you put money towards this, you will not gain reward in proportion. if anything, your net benefit from the cache work goes down if you spend a buck on it. so we have a free rider problem by nature, and thus far only Mercury has actually put up and shut up17:41:08
@edef1c:matrix.orgedefrelative to the benefit received from it, i think the cache is well worth running and well worth running well. but the incentive structure and resulting actual budgets available are not a good time17:42:10
@edef1c:matrix.orgedefwe didn't have data infrastructure for even seeing what was really in it or what it actually runs like until i wrote a bunch of it and coordinated with Jonas to get the AWS access i needed to do that, and while i love teaching people data tooling, data skills have been fairly SPOF-y so far17:44:16
@edef1c:matrix.orgedefif we can get meaningful, recurring funding so people who don't have as many screws loose are motivated to keep the show running, that's music to my ears17:47:54
@whentze:matrix.orgWanja Hentzeis foundation funding even going towards this?17:50:34
@whentze:matrix.orgWanja Hentzea foundation is a way to fund a Commons, right?17:50:51
@edef1c:matrix.orgedefthere is an opencollective thing for the cache sustainability/scalability/preservation project17:51:10
@whentze:matrix.orgWanja Hentzeoh, nice17:51:32
@edef1c:matrix.orgedefbut broadly speaking i am not impressed with how much the (seemingly profitable) ecosystem has contributed towards the Foundation17:51:46
@edef1c:matrix.orgedefthe €5k i've asked for is certainly not enough to cover even my living costs for the amount of time i've sunk in, and i'm not even convinced i should be accepting that money, because the data and its availability isn't safe and sound just yet17:54:00
@edef1c:matrix.orgedefwe have other infrastructural things to improve, Hydra could certainly use work, the build farm in general could use work, i'm just tending to the one that feels like the biggest emergency right now17:56:34
@edef1c:matrix.orgedefsome of these other things will turn into emergencies in due time17:56:55
@edef1c:matrix.orgedef the cache is in rough seas because one sponsor dropped out and we had to hop to the next one, none of these are eternal, same goes for CDNs and build farm compute providers 18:00:23
@edef1c:matrix.orgedefa bunch of the infrastructure governance thus far has been abysmal, and no shade to a bunch of the infra team there, they've been doing their best with what they have18:01:06
@zimbatm:numtide.comJonas Chevalierone idea, and it might be a bad one, but technically we could add a clause on cache.nixos.org says that it can only be used for non-profit work. We don't enforce it, but that helps going to companies and be like: hey, did you see this clause? 18:17:45
@edef1c:matrix.orgedefhmm. i am not a lawyer, and i have mixed feelings about the idea, but i do think "for non-commercial use" could be a reasonable policy18:21:13
@raitobezarius:matrix.orgraitobezariusI think those ideas should be left last last last resort IMHO18:21:33
@raitobezarius:matrix.orgraitobezariusOnce engineering is done, minimal operation costs are secured (including human capital), I think the cost of running such a system is marginal18:22:07
@raitobezarius:matrix.orgraitobezarius Having non commercial use clauses would create weird situations where you like "this package you did for your work in nixpkgs, should it be hydraPlatforms = [];, etc.) 18:22:37
@raitobezarius:matrix.orgraitobezarius * Having non commercial use clauses would create weird situations where you like "this package you did for your work in nixpkgs, should it be hydraPlatforms = [];, etc. 18:22:39
@zimbatm:numtide.comJonas Chevalieryeah it's just an idea. btw, the infra team has been doing a ton of good things lately. my feeling is that we're moving in the right direction overall.18:25:10
@zimbatm:numtide.comJonas Chevalierbut like edef said, there is a lot of old cruft to cleanup18:25:34
@raitobezarius:matrix.orgraitobezariusi think as i put it now18:25:48
@raitobezarius:matrix.orgraitobezarius"let the infra cook"18:25:50
@edef1c:matrix.orgedefpretty much, yeah18:26:25
@edef1c:matrix.orgedefwe can get places, we just need latitude to operate18:26:50

Show newer messages


Back to Room ListRoom Version: 10