| 16 Oct 2025 |
 fzakaria | The bug is about encoding so LocalBinaryCacheStore wouldn't anyways be a 1:1 test | 14:40:21 |
 John Ericson | Oh whoops OK I guess that doesn't help with that then | 14:41:21 |
| fzakaria | although my test doesn't have a fix for it... after writing the unit tests i learned that '+' in the path doesn't get encoded. I couldn't reproduce it; I added notes on the linked issue that maybe it was S3 specific passing it as a query parameter. | 14:43:13 |
 Mic92 | @Ericson2314:matrix.org: https://github.com/Mic92/niks3 since i am not aware of binary caches with gc that support the fully protocol including ca derivations, i created this. Currently working on the second production deployment. Also makes it really cheap to self-host high availability caches. You can get 1tb s3 buckets for 5 to 15 Dollar per month. | 14:46:26 |
| John Ericson | Mic92: cool, but does it depend on harmonia / nixrs ? :) | 14:47:26 |
| Mic92 | * @Ericson2314:matrix.org: https://github.com/Mic92/niks3 since i am not aware of binary caches with gc that support the full protocol including ca derivations, i created this. Currently working on the second production deployment. Also makes it really cheap to self-host high availability caches. You can get 1tb s3 buckets for 5 to 15 Dollar per month. | 14:47:35 |
| Mic92 | No this is a completely different implementation ;) but I promise that the other thing comes next | 14:48:36 |
| Mic92 | The actual difficulty was not the nix related protocols but getting the gc to work with some two phase commit | 14:50:13 |
| John Ericson | Alright, I just wanna start sharing Rust code :) | 14:50:19 |
| Mic92 | All those different object types i just wrote in a couple of days | 14:50:20 |
| Mic92 | * The actual difficulty was not the nix related protocols but getting the gc to work with some two phase commit protocol | 14:50:34 |
| Mic92 | We also should just move to zstd for ls files. It is much faster in my benchmarks compared to brotli. Nix already supports zstd on download for a long time. | 14:54:10 |
 manveru | @Mic92 that looks really nice, we just had an incident where someone turned on object expiration on our bucket to save space... The fallout of narinfos pointing to nothing and Nix crashing because of it took weeks to clean up | 15:13:17 |
| manveru | But does this prevent crashes because of locally cached narinfos that point to removed NARs? | 15:15:41 |
| Mic92 | In reply to @manveru:matrix.org
But does this prevent crashes because of locally cached narinfos that point to removed NARs? I think missing nars for a given narinfo should only produce a warning? At least my experience from with harmonia. Broken references however might be a problem. However niks3 shouldn't let this happen. | 15:18:00 |
| Mic92 | If you don't have gc the second best thing is rotating s3 buckets and have a cdn/load-balancer checking both | 15:19:19 |
| manveru | i'm not 100% certain anymore, but afaict if the NAR goes missing within the narinfo-cache-positive-ttl while there's an entry in the e.g. ~/.cache/nix/binary-cache-v7.sqlite Nix may take it personally | 15:26:50 |
| manveru | but it's possible that's only true for references | 15:27:47 |
| John Ericson | Robert Hensing (roberth): do you know a command to convert yaml to json? | 15:35:34 |
| John Ericson | kinda want the JSON schema to be yaml | 15:35:39 |
| John Ericson | for nicer git diffs | 15:35:42 |
 roberth | yq I guess? You can call me a freak, but I'm a happy JSON editor | 15:39:54 |
| roberth | * John Ericson: yq I guess? You can call me a freak, but I'm a happy JSON editor | 15:40:19 |
| roberth | oof
$ nix-store -qR $(nix-instantiate -A nix) | cut -c 45- | grep -iE 'yq|yaml'
YAML-Tiny-1.74.tar.gz.drv
perl5.40.0-YAML-Tiny-1.74.drv
| 15:43:23 |
| John Ericson | haha ok that's a funny way to find it | 15:46:10 |
 dramforever | yaml2json? | 15:54:18 |
| fzakaria | YAML is a superset of JSON. | 15:56:43 |
| fzakaria | (always blew my mind) | 15:57:11 |
| fzakaria | should we merge https://github.com/NixOS/nix/pull/13673 ?
it's a good test. | 15:58:35 |
| manveru | remarshal is nice, but haven't needed it since switching to nushell ^^; | 16:41:38 |
