| 14 Oct 2025 |
 delroth | makes sense, thanks | 10:12:01 |
 raitobezarius | we can move this over AFNix discussion | 10:12:10 |
| raitobezarius | (https://zulip.afnix.fr/#narrow/channel/4-infra/topic/macOS.20remote.20building.20access for those who wants to follow) | 10:13:03 |
 emily | has there been any more thought given re: builtins.convertHash / https://git.lix.systems/lix-project/lix/issues/602? I know making it a more general radix conversion function was discussed, but IMO the Base32 and Base64/SRI hash formats are quirky enough that they really don't fit elegantly into a radix conversion model, especially since they are too large to round-trip via Nix integers, which would otherwise be the natural way to express radix conversion functions.
not saying it has to have the exact same interface but I am not convinced that the full general base conversion mechanism is necessary/desirable for it, and the -2 review on https://gerrit.lix.systems/c/lix/+/2585 shows that it would require a lot of contortions to make it useful for the actual hash conversion use case, which seems to have been useful both in third-party repos and has caused a hash mismatch in Nixpkgs even with an attempt to do feature detection https://github.com/NixOS/nixpkgs/pull/451929.
| 15:38:52 |
| emily | given it has to deal with >128-bit values and the formats are quirky, I am not sure there is a builtin that is significantly more scoped than builtins.convertHash that would make it possible to implement this efficiently and non-horribly given Nix's anaemic facilities for such processing | 15:40:17 |
| emily | I think the most you could reasonably scope it down is just omitting handling of the SRI stuff and making it do inter-conversion of strings between Base16/Base32/Base64 | 15:41:27 |
| emily | and then handle the <alg>:<digest> and <alg>-<digest> formats in Nix code | 15:41:44 |
| emily | but even then it's not like you'll get something terribly general because e.g. you are still encoding quirks like Nix's Base32 endianness | 15:42:14 |
| emily | so I expect that in ~100% of cases people are just going to be using it for exactly what builtins.convertHash does | 15:42:50 |
| emily | I suppose I can imagine someone trying to convert Base64 binary data to a hex dump or something… and if NUL bytes are going to be allowed in strings then it could be split into two separate encode/decode built-ins that go through binary blobs at a penalty of more allocations etc. | 15:43:44 |
| emily | but the general radix conversion thing seems like a total detour | 15:44:00 |
| raitobezarius | What about {from,to}{NixBase32,Base16,Base64} as primops? | 15:51:13 |
| raitobezarius | Which seems to be that proposal ^ | 15:51:21 |
| raitobezarius | Ideally, the primops should live under a namespace, e.g. conversions or something | 15:51:53 |
| emily | IIRC Base32 is not unambiguously defined for some sizes of byte string. | 16:03:52 |
| emily | so I believe these functions would not round-trip etc. | 16:04:03 |
| emily | I am not sure if you would need to manually add/remove padding in Nix code to make the conversions go through | 16:04:38 |
| emily | it's at least avoided by convertHash taking specification of a hash which restricts the bit format to one of the "standard" ones | 16:04:48 |
| emily | (it would also not be backportable to versions that use a NUL-terminated string representation and would be a permanent(?) divergence from Nix's choice to forbid NULs more thoroughly instead of course, but these may not be issues) | 16:05:51 |
| emily | I am also not entirely sure if there are any gremlins in the <algo>:<digest> format etc. which would be a pain to deal with in Nix | 16:06:15 |
| emily | though it's probably tolerable if it's just in one lib function in Nixpkgs | 16:06:22 |
| raitobezarius | Do you have a clear example of case where {from,to}NixBase32 would not roundtrip? | 16:06:32 |
| raitobezarius | It's already known that Nix flavor of base32 is a bit special, but I cannot remember of seeing there was unambiguous cases. If anything, Nix always accepted $BASE32 + any char for a while (and probably still do). | 16:07:06 |
| emily | frankly the hash formats are all very weird and I am not sure if it is really better to have their format details exposed to userspace rather than just treating them as opaque but interconvertible blogs | 16:07:17 |
| raitobezarius | It never honored any form of strict implementation of RFC4648 | 16:07:22 |
| emily | e.g. proliferation of Base32 use doesn't seem like a great thing | 16:07:32 |
| emily | I believe there are cases where appending (or prepending?) a NUL byte will not change the result | 16:07:40 |
| emily | because of padding | 16:07:45 |
| emily | and therefore decoding must fail to round-trip on one of them | 16:07:56 |
| emily | I recall running into this when factoring out the Base32 functions | 16:08:03 |
