| 16 Oct 2025 |
 Robert Hensing (roberth) | (well, two mostly unrelated schemas) | 14:14:26 |
 John Ericson | yeah was just thinking we might need that | 14:25:36 |
| John Ericson | because of references between then | 14:25:42 |
| John Ericson | but we can wait and see | 14:25:45 |
| Robert Hensing (roberth) | one big schema should also be fine | 14:26:00 |
| Robert Hensing (roberth) | at least for the store layer | 14:26:08 |
| Robert Hensing (roberth) | that's where most of the JSON action is anyway | 14:26:25 |
 fzakaria | John Ericson: i'm not sure what you meant on PR#14266 | 14:40:01 |
| fzakaria | The bug is about encoding so LocalBinaryCacheStore wouldn't anyways be a 1:1 test | 14:40:21 |
| John Ericson | Oh whoops OK I guess that doesn't help with that then | 14:41:21 |
| fzakaria | although my test doesn't have a fix for it... after writing the unit tests i learned that '+' in the path doesn't get encoded. I couldn't reproduce it; I added notes on the linked issue that maybe it was S3 specific passing it as a query parameter. | 14:43:13 |
 Mic92 | @Ericson2314:matrix.org: https://github.com/Mic92/niks3 since i am not aware of binary caches with gc that support the fully protocol including ca derivations, i created this. Currently working on the second production deployment. Also makes it really cheap to self-host high availability caches. You can get 1tb s3 buckets for 5 to 15 Dollar per month. | 14:46:26 |
| John Ericson | Mic92: cool, but does it depend on harmonia / nixrs ? :) | 14:47:26 |
| Mic92 | * @Ericson2314:matrix.org: https://github.com/Mic92/niks3 since i am not aware of binary caches with gc that support the full protocol including ca derivations, i created this. Currently working on the second production deployment. Also makes it really cheap to self-host high availability caches. You can get 1tb s3 buckets for 5 to 15 Dollar per month. | 14:47:35 |
| Mic92 | No this is a completely different implementation ;) but I promise that the other thing comes next | 14:48:36 |
| Mic92 | The actual difficulty was not the nix related protocols but getting the gc to work with some two phase commit | 14:50:13 |
| John Ericson | Alright, I just wanna start sharing Rust code :) | 14:50:19 |
| Mic92 | All those different object types i just wrote in a couple of days | 14:50:20 |
| Mic92 | * The actual difficulty was not the nix related protocols but getting the gc to work with some two phase commit protocol | 14:50:34 |
| Mic92 | We also should just move to zstd for ls files. It is much faster in my benchmarks compared to brotli. Nix already supports zstd on download for a long time. | 14:54:10 |
 manveru | @Mic92 that looks really nice, we just had an incident where someone turned on object expiration on our bucket to save space... The fallout of narinfos pointing to nothing and Nix crashing because of it took weeks to clean up | 15:13:17 |
| manveru | But does this prevent crashes because of locally cached narinfos that point to removed NARs? | 15:15:41 |
| Mic92 | In reply to @manveru:matrix.org
But does this prevent crashes because of locally cached narinfos that point to removed NARs? I think missing nars for a given narinfo should only produce a warning? At least my experience from with harmonia. Broken references however might be a problem. However niks3 shouldn't let this happen. | 15:18:00 |
| Mic92 | If you don't have gc the second best thing is rotating s3 buckets and have a cdn/load-balancer checking both | 15:19:19 |
| manveru | i'm not 100% certain anymore, but afaict if the NAR goes missing within the narinfo-cache-positive-ttl while there's an entry in the e.g. ~/.cache/nix/binary-cache-v7.sqlite Nix may take it personally | 15:26:50 |
| manveru | but it's possible that's only true for references | 15:27:47 |
| John Ericson | Robert Hensing (roberth): do you know a command to convert yaml to json? | 15:35:34 |
| John Ericson | kinda want the JSON schema to be yaml | 15:35:39 |
| John Ericson | for nicer git diffs | 15:35:42 |
| Robert Hensing (roberth) | yq I guess? You can call me a freak, but I'm a happy JSON editor | 15:39:54 |
