| 24 Aug 2023 |
 flokli | It's a conscious decision to have it that granular in Tvix - the idea is that we're able to heavily cache both individual chunks as well as rendered NARs. And long term have the clients be a bit smarter on that front too. | 19:33:50 |
 @brian:bmcgee.ie | You must have some stats now on the benefits/tradeoffs? | 19:34:53 |
| flokli | If the clients themselves use that protocol as their underlying data model, they can cache these files on their own and take care of nar assembly locally, if needed at all. | 19:34:56 |
| flokli | I had some people do some benchmarks on nix-casync, which also chunked whole nar files in its native format, and the chunks were not chunking nicely at file boundaries, often resulting in too large chunks, and shifted / missed dedup possibilitied | 19:35:57 |
 Zhaofeng Li | In the long run, I think this is the way to go if more flexibility is provided. The current Nix isn't very "smart" with regards to binary substitution | 19:36:39 |
| flokli | Again, as written earlier today, I'm aiming for efficient storage and transport, and assume you can cache both individual chunks retrieval and assembled nars if needed. | 19:36:41 |
| Zhaofeng Li | * In the long run, I think this is the way to go if more flexibility is provided. The current Nix client isn't very "smart" with regards to binary substitution | 19:36:46 |
| flokli | Zhaofeng Li: I wanted to start with a http binary cache proxy that speaks tvix-store protocol over the wire and keeps a local cache of chunks and renders nar files just for a local Nix. | 19:37:49 |
| flokli | I'm not sure I want to even expose a NAR interface publicly to clients, they can just substitute through some local proxy. | 19:38:26 |
| flokli | I don't need to provide cache.nixos.org, so I can be a bit more demanding on what clients need to have locally. And most of the time my crappy internet connection is the limiting factor, so having to download less over that is actually both good for my data usage as well as the server hosting the tvix-store, win win ;-) | 19:40:50 |
 raitobezarius | I want fast fast fast | 19:41:25 |
| raitobezarius | I have 5Gbps | 19:41:27 |
| raitobezarius | please give me 5Gbps saturating cache | 19:41:33 |
| flokli | raitobezarius: what about 100G | 19:42:01 |
| Zhaofeng Li | If you have deep visibility/metadata into the store itself, the store itself can even be the chunk cache | 19:42:04 |
| raitobezarius | this I am planning too flokli but it will be when Raito Corp will be done and we will do 400Gbps distribution of the software universe | 19:42:21 |
| raitobezarius | it's surprisingly hard to get OCP to give you a free 400Gbps switch | 19:42:34 |
| Zhaofeng Li | In reply to @raitobezarius:matrix.org
please give me 5Gbps saturating cache Oh nice, I have an appointment to install 5G Google Fiber next month, but my roommate is thinking about bailing out because of the cost | 19:42:38 |
| raitobezarius | In reply to @zhaofeng:zhaofeng.li
Oh nice, I have an appointment to install 5G Google Fiber next month, but my roommate is thinking about bailing out because of the cost well I pay it 30EUR/mo | 19:42:51 |
| flokli | In reply to @zhaofeng:zhaofeng.li
If you have deep visibility/metadata into the store itself, the store itself can even be the chunk cache This is already the case actually - I don't realize the actual files on disk currently, you can look at them through fuse | 19:43:00 |
| raitobezarius | but I support you towards 5G | 19:43:08 |
| flokli | Rust in kernel VFS when | 19:43:10 |
| raitobezarius | In reply to @flokli:matrix.org
Rust in kernel VFS when :DDDDDDDDDDDDDDDDDD | 19:43:21 |
| Zhaofeng Li | In reply to @flokli:matrix.org
This is already the case actually - I don't realize the actual files on disk currently, you can look at them through fuse Very cool, didn't know that! | 19:44:00 |
| raitobezarius | https://cs.tvl.fyi/depot/-/blob/tvix/store/src/fuse/mod.rs | 19:44:23 |
| raitobezarius | if I'm not wrong | 19:44:45 |
| flokli | It's all a bit wip, there's no seek support, because I want to do this in concert with verified streaming | 19:45:12 |
| flokli | But it should happen all in the next months | 19:45:30 |
| flokli | But it should already be good enough for builds | 19:46:51 |
| flokli | I'll talk about it during nixcom | 19:47:24 |
