| 30 Jul 2025 |
 John Ericson | like, as soon as you have string tags, there is probably some slack like is lexigraphic ordering required | 15:14:07 |
| John Ericson | * like, as soon as you have string tags, there is probably some slack like whether lexigraphic ordering required | 15:14:16 |
 fzakaria | i'm not usually a big protobuf fan but i do like having a high level IDL to define the thigns being stored. | 15:15:11 |
| John Ericson | if/when we stop putting drvs in store objects and go full harvard architecture, we can no worry about the on-disk version being the hashed canonical form | 15:15:13 |
 magic_rb | Sorry to interject, whats BCS | 15:16:00 |
| John Ericson | but so long as we're deciding "what goes in the drv file", the drvPath is that hash of that | 15:16:02 |
| John Ericson | magic_rb: from facebook's blockchain, it is basically concat the things to do products, use leb128 or whatever the variable thing is called to do enum tags for sums | 15:16:40 |
| John Ericson | it is a formalization of sort of the "obvious" way to do binary representations of algebraic data types | 15:17:12 |
| John Ericson | no string tags, no self description, everything is ordered (products have a left side and right side, the variants of the sum types are totally ordered and assigned 0 1 2 3 4....) | 15:17:52 |
| John Ericson | it is basically what you would get on the heap too, except there is no padding/alignment, so you can't just cast it captn-proto style in general | 15:18:19 |
| John Ericson | * it is basically what you would get on the heap (if no pointers) too, except there is no padding/alignment, so you can't just cast it captn-proto style in general | 15:19:00 |
| magic_rb | Ill read up more on it, its good to know about this stuff, for when you need a binary format | 15:20:03 |
 emily | yeah. there are more and less rigorous standards about it though. (e.g. https://preserves.dev/canonical-binary.html again as an exemplar.) but anyway I'd just be repeating what I said overviewing a whole bunch of formats last time I think :) | 15:21:33 |
| emily | FWIW protobufs are worse at canonicalization than most formats IIRC | 15:22:05 |
| emily | https://protobuf.dev/programming-guides/serialization-not-canonical/ | 15:22:14 |
| emily | "Inherent Barriers to Stable Serialization" :) | 15:22:27 |
| fzakaria | makes sense; i would probably calculate my own stable key | 15:23:42 |
| fzakaria | * makes sense; i would probably calculate my own stable key off the requisite fields | 15:23:53 |
| fzakaria | i have rarely ever used the serialization format itself as inputs to things | 15:24:37 |
| John Ericson | is this preserves thing widely implemented? | 15:24:44 |
| fzakaria | but maybe its because i never have canonicalization :P | 15:24:50 |
| John Ericson | I would like to make it easier to implement "nixlets" | 15:25:00 |
| emily | no, as I said ecosystem is the main drawback. it's being used in the OCapN effort to standardize CapTP between Agoric/Spritely/I think Cap'n Proto too, and it's used in https://syndicate-lang.org, but it's not the kind of thing you'd expect to find a library in any random language for. I'm not sure what the "second-best" thing is. maybe "whichever of the several CBOR canonical forms is what CTAP uses", but IIRC that's specifically not the RFC one, so that's a nightmare.
I don't know about BCS but at a glance it seems like it does a reasonable job, assuming everyone agrees on what the format actually is. it's not self-describing though, so extensibility will indeed suffer
| 15:28:27 |
| John Ericson | yeah I don't think we can use BCS | 15:29:18 |
| John Ericson | it would not be good to use a map for an extensible struct | 15:30:44 |
| John Ericson | because one then gets unnecessary enum tags on the values, which hinders extensibility | 15:31:00 |
| John Ericson | (you can freely choose your own not-yet-used key, but you need to also choose a not-yet-used enum tag, which is harder) | 15:31:23 |
| magic_rb | NDBF - Nix Derivation Binary Format | 17:00:42 |
 flokli | https://xkcd.com/927/ | 22:35:11 |
| 31 Jul 2025 |
|  AshKaashh joined the room. | 22:51:16 |
