| 13 Jul 2024 |
 Ronny | In reply to @samrose:matrix.org
Ronny: one route is to just use it as a remote build machine https://nix.dev/manual/nix/2.18/advanced-topics/distributed-builds. Nix would automatically cache the builds in the nix store of the machine. you can nix build with --keep-outputs to build up the roots you want to keep. im building on one of my higher powered systems and have the cache on my rpi sytem (that shouldn't build) - i want to work out something where a github action pushes to the cache eventually | 10:19:31 |
| 14 Jul 2024 |
 samrose | Ronny: that is explained in https://nix.dev/manual/nix/2.22/command-ref/new-cli/nix3-copy you just have to figure out how to securely network to your machine. | 10:21:20 |
| Ronny | In reply to @samrose:matrix.org
Ronny: that is explained in https://nix.dev/manual/nix/2.22/command-ref/new-cli/nix3-copy you just have to figure out how to securely network to your machine. this lacks all details on gc-proofing and/or lifecycle management - copied closures dont exactly help with controlling how one reclaims space | 10:31:16 |
 hexa | have you looked at attic so far? | 11:41:25 |
| hexa | it is a binary cache implementation that uses S3-compatible backends | 11:42:13 |
| hexa | https://github.com/zhaofengli/attic | 11:42:15 |
| hexa | gc is documented here https://docs.attic.rs/tutorial.html#garbage-collection | 11:42:34 |
| hexa | i'm using it with a local garage setup, and the same would likely work with minio, or some hosted setup | 11:44:02 |
| hexa | if you're interested I can share the config that I run on my local rock5b | 11:50:09 |
| Ronny | i have taken a look at attic, however as my server has also the largest deploy set, i was hoping to enable using its store directly as a cache instead of creating a extra deployed service (when ssh and filesystem are kinda there) | 13:14:43 |
| hexa | you can use harmonia to serve your /nix/store | 13:18:33 |
| hexa | * you can use harmonia to serve your /nix/store | 13:18:44 |
| hexa | it has no gc semantics beyond what the nix store already offers | 13:19:12 |
| Ronny | hmm, man, so unless i use a different cache with a different store, i get no gc for the system that wont nke cache by accident | 13:21:57 |
| hexa | one tricky thing about nix and caches that collect garbage is a very high postive ttl for narinfo files in nix's defaults https://nix.dev/manual/nix/2.18/command-ref/conf-file#conf-narinfo-cache-positive-ttl | 13:22:04 |
| hexa | * one tricky thing about nix and caches that collect garbage is a very high postive ttl (30 days) for narinfo files in nix's defaults https://nix.dev/manual/nix/2.18/command-ref/conf-file#conf-narinfo-cache-positive-ttl | 13:22:52 |
| hexa | * one tricky thing about nix and caches that collect garbage is a very high postive ttl (30 days) for narinfo files in its defaults https://nix.dev/manual/nix/2.18/command-ref/conf-file#conf-narinfo-cache-positive-ttl | 13:23:02 |
| hexa | well, you could manage gcroots for the closures you want to keep | 13:23:48 |
| hexa | this is basically the same mechanic that prevents gc from removing things that shouldn't be removed | 13:24:35 |
| hexa | * this is basically the same mechanic that prevents gc from removing things that shouldn't be removed, like your system closure | 13:24:42 |
| hexa | * this is basically the same mechanic that prevents gc from removing things that shouldn't be removed, like your recent system closures | 13:24:54 |
| Ronny | so based on what i understood so far - there is no nix-native binary cache mechanism - gc will jsut kill unless one does messy gc root hacks | 13:26:12 |
| hexa | correct | 13:26:43 |
| Ronny | I truly wonder why,, it seems like that would be the easiest way for a individual or a small org to handle things | 13:32:31 |
| hexa | I wouldn't be surprised to find out that remote builds were just invented for hydra 🙂 | 13:36:28 |
 Jonas Chevalier | TIL; there are now 61TB SSDs in the professional market.
https://hothardware.com/reviews/solidigm-d5-p5336-review-61tb-data-center-ssd | 16:46:49 |
| Jonas Chevalier | According to Solidigm's marketing material, the TCO for 1PB of storage is only 28k/year! | 16:47:33 |
| Jonas Chevalier | That would fit the whole binary cache, including some redundant copy | 16:47:50 |
| 17 Jul 2024 |
| samrose | In reply to @ronnypfannschmidt:matrix.org
I truly wonder why,, it seems like that would be the easiest way for a individual or a small org to handle things Most of the cases where I worked at orgs that use nix, (small to medium sized) we would maintain binary cache in s3 compatible object store. It's low cost, eliminates the problem of garbage collection or maintaining a specific machine, and is very simple to make the cache widely available for deployed machines, ci, etc Once I started doing it this way, I never looked back.
I did have a former employer that tried to manage a running machine as a nix binary cache. It didn't go very well, although it's an approach that they used for a number of years. It took a lot of extra work to maintain the machine that served that purpose.
| 03:07:21 |
 @elvishjerricco:matrix.org | anyone here using hydra to build / cache staging for your own systems locally? Wondering just how much extra churn that causes normally and if it's worth it to be able to build / test on staging | 16:49:48 |
