| 24 Aug 2023 |
 @brian:bmcgee.ie | A PATCH would technically be the 'correct' way to do this I guess | 15:27:29 |
| @brian:bmcgee.ie | Or it looks like ETag and If-Match is how you can prevent this https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag#avoiding_mid-air_collisions | 15:28:56 |
| @brian:bmcgee.ie | Nix client would refresh and try again | 15:29:34 |
 @linus:schreibt.jetzt | I think generally something like attic is a better approach to a binary cache anyway | 15:29:51 |
| @brian:bmcgee.ie | attic is using it's own client right? | 15:31:25 |
| @brian:bmcgee.ie | instead of the http interface | 15:31:30 |
| @linus:schreibt.jetzt | it can serve Nix's protocol read-only as well | 15:31:44 |
| @brian:bmcgee.ie | Makes sense now why Domen did it aswell | 15:32:00 |
| @brian:bmcgee.ie | the http interface is fine for read | 15:32:11 |
| @brian:bmcgee.ie | not that great for the writes | 15:32:18 |
| @brian:bmcgee.ie | For context, I'm playing around with my own cache implementation which is why I'm interested | 15:33:05 |
| @linus:schreibt.jetzt | one thing I'd like to have in Attic is the ability to have signatures from the client in addition to server-managed signing keys | 15:34:12 |
| @brian:bmcgee.ie | Have it merge on upload? | 15:34:34 |
| @linus:schreibt.jetzt | well currently attic just stores a single signature for each narinfo (which is a database table entry, not an actual file), and that signature is generated by the server at upload time | 15:36:02 |
| @linus:schreibt.jetzt | but it means that you have to trust the server much more than if the client could provide its own signatures | 15:36:24 |
| @brian:bmcgee.ie | makes sense, also what happens with key rotation on the server | 15:37:06 |
| @linus:schreibt.jetzt | key rotation is generally a tricky topic with nix https://github.com/NixOS/rfcs/pull/149 | 15:37:58 |
 Zhaofeng Li | In reply to @linus:schreibt.jetzt
well currently attic just stores a single signature for each narinfo (which is a database table entry, not an actual file), and that signature is generated by the server at upload time The signature is generated at download time using the the per-cache private key managed by the server. It does store client-supplied signatures in the database but they aren't exposed at the moment. I wrote a bit more here: https://github.com/zhaofengli/attic/issues/80#issuecomment-1684347741
(oh right, forgot to respond)
| 15:40:33 |
| Zhaofeng Li | So we can easily have client-managed signatures now by changing how narinfo is generated (need to fix serialization for multiple signatures), but I'd prefer to have a complete story UX-wise (the client should be able to automatically sign on upload) | 15:42:27 |
| Zhaofeng Li | In reply to @brian:bmcgee.ie
makes sense, also what happens with key rotation on the server You currently can rotate the server-managed key with attic cache configure --regenerate-keypair, but all clients who download need to reconfigure their trusted public keys | 15:44:35 |
| Zhaofeng Li | So the resulting experience can be frustrating | 15:44:56 |
| @linus:schreibt.jetzt | In reply to @zhaofeng:zhaofeng.li
The signature is generated at download time using the the per-cache private key managed by the server. It does store client-supplied signatures in the database but they aren't exposed at the moment. I wrote a bit more here: https://github.com/zhaofengli/attic/issues/80#issuecomment-1684347741
(oh right, forgot to respond)
oops, sorry for talking nonsense 🙃 | 15:54:44 |
| @brian:bmcgee.ie | Am I right in thinking, that with a custom client for pushing to the cache, that's where you're calculating the chunks / doing the heavy lifting for the de-duplication? Zhaofeng Li: | 17:57:05 |
| @brian:bmcgee.ie | Operating on the uncompressed store paths instead of having to decompress them on the server side | 17:57:43 |
| Zhaofeng Li | In reply to @brian:bmcgee.ie
Am I right in thinking, that with a custom client for pushing to the cache, that's where you're calculating the chunks / doing the heavy lifting for the de-duplication? Zhaofeng Li: Actually deduplication is handled server-side and is transparent to the client. As a result, the upload API is very simple and the client just streams the entire uncompressed NAR to the server | 19:09:49 |
| @brian:bmcgee.ie | In reply to @zhaofeng:zhaofeng.li
Actually deduplication is handled server-side and is transparent to the client. As a result, the upload API is very simple and the client just streams the entire uncompressed NAR to the server 🤔 | 19:15:05 |
| @brian:bmcgee.ie | So you dedupe the nar archive not the uncompressed form? | 19:15:43 |
| @brian:bmcgee.ie | Naively I would have thought it better to operate on the uncompressed store path | 19:16:25 |
| @brian:bmcgee.ie | Ah wait, does nar offer any compression? Is that why you specify zstd or xz when working with the cache. | 19:18:24 |
| @brian:bmcgee.ie | * Ah wait, does nar offer any compression? Is that why you specify zstd or xz when working with a cache. | 19:18:52 |
