!CcTBuBritXGywOEGWJ:matrix.org

NixOS Binary Cache Self-Hosting

170 Members
About how to host a very large-scale binary cache and more58 Servers

Load older messages


SenderMessageTime
24 Aug 2023
@flokli:matrix.orgflokliso YMMV10:15:41
@linus:schreibt.jetzt@linus:schreibt.jetztoh right, yeah, if it's just a CDN without extra smarts that makes sense10:15:56
@linus:schreibt.jetzt@linus:schreibt.jetztbut depending on how costly requests are, it could make sense to have caching both behind and in front of the assembly bit10:16:28
@flokli:matrix.orgflokliI still think you should optimize for disk storage on S3 / $backingStore, so small chunk sizes, because everything in the front is cache-able.10:16:33
@linus:schreibt.jetzt@linus:schreibt.jetztat a large scale at least10:16:34
@brian:bmcgee.ieBMG joined the room.14:55:34
@brian:bmcgee.ieBMGHey, I've been looking into the binary cache protocol today and have noticed that once you do a copy, you can never update the narinfo again. If you sign the path locally with a new key and want to push it, well you can't. 15:02:47
@brian:bmcgee.ieBMGFound these related issues https://github.com/NixOS/nix/issues/4221 https://github.com/NixOS/nix/issues/756215:02:56
@brian:bmcgee.ieBMGAm I right?15:03:03
@linus:schreibt.jetzt@linus:schreibt.jetzt BMG: there's a dedicated nix store copy-sigs command, I wonder if that works? 15:08:54
@brian:bmcgee.ieBMGnot that i've been able to make work yet15:11:03
@linus:schreibt.jetzt@linus:schreibt.jetztok, then I'm not sure. But yeah there are a lot of weird things about narinfos in flat-file binary caches15:11:45
@linus:schreibt.jetzt@linus:schreibt.jetzt(also paths only being able to have one deriver is weird in general)15:12:22
@brian:bmcgee.ieBMG
In reply to @linus:schreibt.jetzt
BMG: there's a dedicated nix store copy-sigs command, I wonder if that works?
It seems to be geared towards copying signatures from a remote store into your local store. Can't see a way of copying from local to remote
15:12:45
@linus:schreibt.jetzt@linus:schreibt.jetzt might work if you pass --store file:///... 15:13:18
@linus:schreibt.jetzt@linus:schreibt.jetzt(or s3:/// or whatever, as appropriate)15:13:27
@brian:bmcgee.ieBMGThat means copying from that store into your local. I'm looking at updating a remote cache after i've signed something again locally15:13:55
@linus:schreibt.jetzt@linus:schreibt.jetzt no, --store is the "destination" store 15:14:19
@brian:bmcgee.ieBMGWell I don't have a use case, just wanted to confirm that uploading a narinfo is a one and done action. You have to remove it remotely in order to upload again15:14:20
@linus:schreibt.jetzt@linus:schreibt.jetzt--substituter is where it's copied from15:14:33
@brian:bmcgee.ieBMGimage.png
Download image.png
15:14:40
@linus:schreibt.jetzt@linus:schreibt.jetzt to be clear, I'm not sure if it actually works, but using --store should be the right way to express what you want 15:15:39
@brian:bmcgee.ieBMGjust playing around with it to see15:15:52
@brian:bmcgee.ieBMG nix store copy-sigs -s http://localhost:3000 nixpkgs#hello --refresh --debug is showing the following 15:17:23
@brian:bmcgee.ieBMG
downloading 'http://localhost:3000/ibpsas4imhv84qmdk5ffh51y0ayrqa94.narinfo'...
starting download of http://localhost:3000/ibpsas4imhv84qmdk5ffh51y0ayrqa94.narinfo
finished download of 'http://localhost:3000/ibpsas4imhv84qmdk5ffh51y0ayrqa94.narinfo'; curl status = 0, HTTP status = 200, body = 1344 bytes
imported 0 signatures
download thread shutting down
15:17:35
@linus:schreibt.jetzt@linus:schreibt.jetzt --store, not --substituter 15:17:42
@linus:schreibt.jetzt@linus:schreibt.jetzt -s is --substituter 15:17:50
@brian:bmcgee.ieBMGah ... fuck me15:17:51
@brian:bmcgee.ieBMG nix sign-paths --store <store> -k <private key> <path> from the first issue was the one that worked. Copy didn't seem to 15:22:11
@linus:schreibt.jetzt@linus:schreibt.jetzt yeah, I'm not surprised at nix copy not changing the narinfo 15:23:53

Show newer messages


Back to Room ListRoom Version: 10