!CcTBuBritXGywOEGWJ:matrix.org

NixOS Binary Cache Self-Hosting

109 Members
About how to host a very large-scale binary cache and more48 Servers

Load older messages


SenderMessageTime
14 Jul 2024
@ronnypfannschmidt:matrix.orgRonny I truly wonder why,, it seems like that would be the easiest way for a individual or a small org to handle things 13:32:31
@hexa:lossy.networkhexaI wouldn't be surprised to find out that remote builds were just invented for hydra 🙂 13:36:28
@zimbatm:numtide.comJonas ChevalierTIL; there are now 61TB SSDs in the professional market. https://hothardware.com/reviews/solidigm-d5-p5336-review-61tb-data-center-ssd16:46:49
@zimbatm:numtide.comJonas ChevalierAccording to Solidigm's marketing material, the TCO for 1PB of storage is only 28k/year!16:47:33
@zimbatm:numtide.comJonas ChevalierThat would fit the whole binary cache, including some redundant copy16:47:50
17 Jul 2024
@samrose:matrix.orgsamrose
In reply to @ronnypfannschmidt:matrix.org
I truly wonder why,, it seems like that would be the easiest way for a individual or a small org to handle things

Most of the cases where I worked at orgs that use nix, (small to medium sized) we would maintain binary cache in s3 compatible object store. It's low cost, eliminates the problem of garbage collection or maintaining a specific machine, and is very simple to make the cache widely available for deployed machines, ci, etc Once I started doing it this way, I never looked back.

I did have a former employer that tried to manage a running machine as a nix binary cache. It didn't go very well, although it's an approach that they used for a number of years. It took a lot of extra work to maintain the machine that served that purpose.

03:07:21
@elvishjerricco:matrix.orgElvishJerriccoanyone here using hydra to build / cache staging for your own systems locally? Wondering just how much extra churn that causes normally and if it's worth it to be able to build / test on staging16:49:48
@elvishjerricco:matrix.orgElvishJerricco * anyone here buliding / caching staging for your own systems locally? Wondering just how much extra churn that causes normally and if it's worth it to be able to build / test on staging16:50:27
18 Jul 2024
@sofo:matrix.orgSofi changed their display name from Sofie to Sofie (AWAY).16:37:39
22 Jul 2024
@brokenpip3:matrix.orgbrokenpip3 joined the room.07:47:45
@sinavir:matrix.org@sinavir:matrix.org left the room.09:51:54
23 Jul 2024
@ezzobirbezziou:matrix.orgEzzobir Bezziou joined the room.08:21:01
26 Jul 2024
@71rd:catgirl.cloud71rd joined the room.11:44:15
@joerg:thalheim.ioMic92 joined the room.14:46:02
3 Aug 2024
@grw00:matrix.orggrw00 joined the room.15:10:23
7 Aug 2024
@quapka4:matrix.orgquapka4 joined the room.09:51:03
@quapka4:matrix.orgquapka4Hi folks, can a binary cache be shared for NixOS and say Arch + Nix?09:53:25
@qyliss:fairydust.spaceAlyssa Rossyes09:54:14
@quapka4:matrix.orgquapka4 Oh, I guess the Arch + Nix can still use /nix/store so hashes of the paths match, right? 09:54:48
@qyliss:fairydust.spaceAlyssa Rosscorrect09:54:54
@quapka4:matrix.orgquapka4 Then a follow up question. I am thinking of having a slow virtual machine with the binary cache that fetches the binaries from much beefier - however, I don't have root there. Can I use the nix-user-chroot or something similar to still build as if I was root and had the access to /nix/store? 09:56:19
@quapka4:matrix.orgquapka4
In reply to @qyliss:fairydust.space
correct
Great, thanks.
09:56:25
@aloisw:kde.org@aloisw:kde.org
In reply to @quapka4:matrix.org
Then a follow up question. I am thinking of having a slow virtual machine with the binary cache that fetches the binaries from much beefier - however, I don't have root there. Can I use the nix-user-chroot or something similar to still build as if I was root and had the access to /nix/store?
In principle building without root access is possible, but keep in mind that the single-user sandbox currently has some shortcomings that increase the chance for impurities.
10:55:00
@quapka4:matrix.orgquapka4Thanks, I am not aware of that.10:55:34
8 Aug 2024
@ronnypfannschmidt:matrix.orgRonny

Is there a easy Way to have per user profiles on a server with custom gc

I'm trying to simulate sensible gc for a ssh based cache by creating per flake output profiles that gc via older than similar to deployed profiles

13:40:09
@qyliss:fairydust.spaceAlyssa Ross set a profile picture.18:48:41
11 Aug 2024
@joerg:thalheim.ioMic92Just put them in /nix/var/nix/profiles/per-user05:31:22
@joerg:thalheim.ioMic92and have a script that creates/deletes gcroots based on some heuristic.05:31:59
@ronnypfannschmidt:matrix.orgRonny Mic92: aware of any example of pushing a derivation to a server and then adding a g root for it to that server, the concrete mechanics are unclear to me 18:43:26
@joerg:thalheim.ioMic92I would write a shell script that does nix copy and than register like this: https://nix.dev/manual/nix/2.18/package-management/garbage-collector-roots.html18:48:12

Show newer messages


Back to Room ListRoom Version: 10