!CcTBuBritXGywOEGWJ:matrix.org

NixOS Binary Cache Self-Hosting

139 Members
About how to host a very large-scale binary cache and more49 Servers

Load older messages


SenderMessageTime
14 Jul 2024
@hexa:lossy.networkhexa * you can use harmonia to serve your /nix/store 13:18:44
@hexa:lossy.networkhexait has no gc semantics beyond what the nix store already offers13:19:12
@ronnypfannschmidt:matrix.orgRonnyhmm, man, so unless i use a different cache with a different store, i get no gc for the system that wont nke cache by accident13:21:57
@hexa:lossy.networkhexaone tricky thing about nix and caches that collect garbage is a very high postive ttl for narinfo files in nix's defaults https://nix.dev/manual/nix/2.18/command-ref/conf-file#conf-narinfo-cache-positive-ttl13:22:04
@hexa:lossy.networkhexa * one tricky thing about nix and caches that collect garbage is a very high postive ttl (30 days) for narinfo files in nix's defaults https://nix.dev/manual/nix/2.18/command-ref/conf-file#conf-narinfo-cache-positive-ttl13:22:52
@hexa:lossy.networkhexa * one tricky thing about nix and caches that collect garbage is a very high postive ttl (30 days) for narinfo files in its defaults https://nix.dev/manual/nix/2.18/command-ref/conf-file#conf-narinfo-cache-positive-ttl13:23:02
@hexa:lossy.networkhexawell, you could manage gcroots for the closures you want to keep13:23:48
@hexa:lossy.networkhexathis is basically the same mechanic that prevents gc from removing things that shouldn't be removed13:24:35
@hexa:lossy.networkhexa * this is basically the same mechanic that prevents gc from removing things that shouldn't be removed, like your system closure13:24:42
@hexa:lossy.networkhexa * this is basically the same mechanic that prevents gc from removing things that shouldn't be removed, like your recent system closures13:24:54
@ronnypfannschmidt:matrix.orgRonnyso based on what i understood so far - there is no nix-native binary cache mechanism - gc will jsut kill unless one does messy gc root hacks13:26:12
@hexa:lossy.networkhexacorrect13:26:43
@ronnypfannschmidt:matrix.orgRonny I truly wonder why,, it seems like that would be the easiest way for a individual or a small org to handle things 13:32:31
@hexa:lossy.networkhexaI wouldn't be surprised to find out that remote builds were just invented for hydra 🙂 13:36:28
@zimbatm:numtide.comJonas ChevalierTIL; there are now 61TB SSDs in the professional market. https://hothardware.com/reviews/solidigm-d5-p5336-review-61tb-data-center-ssd16:46:49
@zimbatm:numtide.comJonas ChevalierAccording to Solidigm's marketing material, the TCO for 1PB of storage is only 28k/year!16:47:33
@zimbatm:numtide.comJonas ChevalierThat would fit the whole binary cache, including some redundant copy16:47:50
17 Jul 2024
@samrose:matrix.orgsamrose
In reply to @ronnypfannschmidt:matrix.org
I truly wonder why,, it seems like that would be the easiest way for a individual or a small org to handle things

Most of the cases where I worked at orgs that use nix, (small to medium sized) we would maintain binary cache in s3 compatible object store. It's low cost, eliminates the problem of garbage collection or maintaining a specific machine, and is very simple to make the cache widely available for deployed machines, ci, etc Once I started doing it this way, I never looked back.

I did have a former employer that tried to manage a running machine as a nix binary cache. It didn't go very well, although it's an approach that they used for a number of years. It took a lot of extra work to maintain the machine that served that purpose.

03:07:21
@elvishjerricco:matrix.org@elvishjerricco:matrix.organyone here using hydra to build / cache staging for your own systems locally? Wondering just how much extra churn that causes normally and if it's worth it to be able to build / test on staging16:49:48
@elvishjerricco:matrix.org@elvishjerricco:matrix.org * anyone here buliding / caching staging for your own systems locally? Wondering just how much extra churn that causes normally and if it's worth it to be able to build / test on staging16:50:27
18 Jul 2024
@sofo:matrix.orgSofie changed their display name from Sofie to Sofie (AWAY).16:37:39
22 Jul 2024
@brokenpip3:matrix.orgbrokenpip3 joined the room.07:47:45
@sinavir:matrix.org@sinavir:matrix.org left the room.09:51:54
23 Jul 2024
@ezzobirbezziou:matrix.orgEzzobir Bezziou joined the room.08:21:01
26 Jul 2024
@71rd:catgirl.cloud71rd joined the room.11:44:15
@joerg:thalheim.ioMic92 joined the room.14:46:02
3 Aug 2024
@grw00:matrix.orggrw00 joined the room.15:10:23
7 Aug 2024
@quapka4:matrix.orgquapka4 joined the room.09:51:03
@quapka4:matrix.orgquapka4Hi folks, can a binary cache be shared for NixOS and say Arch + Nix?09:53:25
@qyliss:fairydust.spaceAlyssa Rossyes09:54:14

Show newer messages


Back to Room ListRoom Version: 10