!CcTBuBritXGywOEGWJ:matrix.org

NixOS Binary Cache Self-Hosting

168 Members
About how to host a very large-scale binary cache and more57 Servers

Load older messages


SenderMessageTime
13 Feb 2024
@flokli:matrix.orgflokli If we self-host the binary cache, we probably won't add our own billing department 15:11:04
@edef1c:matrix.orgedef
In reply to @artemist:mildlyfunctional.gay
On the topic of attic, are there places people suggest hosting the backing objects? I've been using b2 but find it very unreliable and slow
yeah backblaze is absolute clown shoes, it just be like that
15:11:20
@edef1c:matrix.orgedef
In reply to @flokli:matrix.org
If we self-host the binary cache, we probably won't add our own billing department
i'm afraid disk iops are still finite resources no matter how you slice it
15:11:44
@julienmalka:matrix.orgJulien
In reply to @flokli:matrix.org
If we self-host the binary cache, we probably won't add our own billing department
Is this on the table you think ?
15:12:36
@flokli:matrix.orgflokliAck, but I don't think the disk iops will be the problem, the actual traffic we egress from the bucket is fairly slim15:12:44
@edef1c:matrix.orgedeflike, deeply depends what our SLOs are here ofc15:12:47
@raitobezarius:matrix.orgraitobezarius
In reply to @flokli:matrix.org
Ack, but I don't think the disk iops will be the problem, the actual traffic we egress from the bucket is fairly slim
80TB/mo if my memory serve me well
15:13:13
@edef1c:matrix.orgedefhow much we want to rely on Fastly as frontend cache etc15:13:17
@raitobezarius:matrix.orgraitobezarius
In reply to @flokli:matrix.org
Ack, but I don't think the disk iops will be the problem, the actual traffic we egress from the bucket is fairly slim
* 60(80TB/mo if my memory serve me well
15:13:20
@raitobezarius:matrix.orgraitobezarius * 60-80TB/mo if my memory serve me well 15:13:22
@flokli:matrix.orgflokliFor now I don't think we can solve both problems, let's solve the bucket problem first 15:14:01
@flokli:matrix.orgflokliPropagating some metadata, and being a bit smarter with the traffic at the CDN level is also something we can tackle, but that requires smarter clients15:14:38
14 Feb 2024
@edef1c:matrix.orgedefso: flipside to Backblaze10:27:55
@edef1c:matrix.orgedefthey'll cover our egress costs if we commit to them, and they are vastly cheaper https://twitter.com/JakeDChampion/status/175750882068997362710:28:08
@edef1c:matrix.orgedefand we get free bandwidth to Fastly10:28:26
@edef1c:matrix.orgedef$36k/yr = $3k/mo, and no more bandwidth charges10:28:46
15 Feb 2024
@hexa:lossy.networkhexaI have four remote builders. Is there a simple way to have them share their store between each other for substitutiion?03:48:49
@zimbatm:numtide.comJonas Chevalierhttps://github.com/cid-chan/peerix maybe? I haven't tested it out yet09:08:38
@zimbatm:numtide.comJonas Chevalieror setup https://github.com/nix-community/harmonia on each node and configure the caches if they are static?09:10:28
@linus:schreibt.jetzt@linus:schreibt.jetzt
In reply to @hexa:lossy.network
I have four remote builders. Is there a simple way to have them share their store between each other for substitutiion?
I think adding the others to substituters as ssh-ng stores would work, but I'm not sure how well (might perform terribly)
11:53:30
@hexa:lossy.networkhexathanks! peerix sounds like the simplest solution, if it (still) works?12:03:38
@hexa:lossy.networkhexa * thanks! peerix sounds like the simplest solution, if it (still) works.12:03:40
@hexa:lossy.networkhexabut seems to suffer from timeouts12:04:42
@a-kenji:matrix.orga-kenji joined the room.19:15:09
19 Feb 2024
@rhizomes:matrix.orgrhizomes joined the room.04:28:19
20 Feb 2024
@sofo:matrix.orgSofie changed their display name from Sofi to Sofie.07:39:16
@sofo:matrix.orgSofie changed their profile picture.14:39:13
@sofo:matrix.orgSofie changed their profile picture.14:41:03
@sofo:matrix.orgSofie changed their profile picture.14:42:35
@sofo:matrix.orgSofie changed their profile picture.14:43:54

Show newer messages


Back to Room ListRoom Version: 10