!CcTBuBritXGywOEGWJ:matrix.org

NixOS Binary Cache Self-Hosting

172 Members
About how to host a very large-scale binary cache and more58 Servers

You have reached the beginning of time (for this room).


SenderMessageTime
23 Aug 2023
@linus:schreibt.jetzt@linus:schreibt.jetztmaybe I should just dig into it rather than putting every intermediate thought in here :p12:06:39
@raitobezarius:matrix.orgraitobezarius max_connections is the right knob 12:06:51
@linus:schreibt.jetzt@linus:schreibt.jetztI don't think this is on the postgres side, it's on the attic side12:07:15
@raitobezarius:matrix.orgraitobezariusAh right I agree12:07:27
@raitobezarius:matrix.orgraitobezariushttps://github.com/NixOS/nixos-org-configurations/tree/master/metrics/fastly13:19:45
@raitobezarius:matrix.orgraitobezariusWe kinda have weak popcount13:20:49
24 Aug 2023
@joerg:thalheim.ioMic92I actually wonder if attic is not massively increasing the s3 costs over just downloading nars because it splits up nars into smaller files that can be deduplicated. You pay also per request and not just for the bandwidth.06:53:29
@joerg:thalheim.ioMic92I also noticed when using local storage it will put all objects in one directory. This might scale badly for large number of files. It's better to create subdirectories based on the filename prefix.06:55:28
@andreas.schraegle:helsinki-systems.de@andreas.schraegle:helsinki-systems.de
In reply to @joerg:thalheim.io
I actually wonder if attic is not massively increasing the s3 costs over just downloading nars because it splits up nars into smaller files that can be deduplicated. You pay also per request and not just for the bandwidth.
that depends on your s3 provider
08:10:33
@raitobezarius:matrix.orgraitobezarius
In reply to @joerg:thalheim.io
I actually wonder if attic is not massively increasing the s3 costs over just downloading nars because it splits up nars into smaller files that can be deduplicated. You pay also per request and not just for the bandwidth.
we can run simulations against AWS S3
08:57:53
@raitobezarius:matrix.orgraitobezariusI am running attic seriously yet so I don't know how easy this is08:58:03
@raitobezarius:matrix.orgraitobezarius * I am not running attic seriously yet so I don't know how easy this is08:58:06
@flokli:matrix.orgflokliThis stuff heavily depends on how good your caching in front of this is10:12:16
@flokli:matrix.orgflokliOf course, if we don't cache and always reach out to S3 to always assemble stuff it'll be more requests, no surprise10:12:40

Show newer messages


Back to Room ListRoom Version: 10