| 20 Mar 2026 |
 emily | the fact that JSON is a bad format doesn't mean Nix shouldn't be able to parse JSON | 10:12:15 |
| emily | having a parseJSONWith that lets you be more specific about how to handle weird issues might be good, but is a separate matter | 10:12:32 |
 KFears& 🏳️⚧️ (they/them) | We prefer the behavior of taking the last key's value but parsing successfully otherwise, in a general-case JSON implementation. Because we consider JSON with duplicate keys to be malformed, but not to a degree where you'd reject it outright, without options to parse it more liberally | 10:13:34 |
| emily | even going by that blog post it's very very rare for implementations to reject duplicate keys | 10:14:02 |
| emily | though sadly it doesn't look like they checked how it's resolved for differing values of the same key | 10:14:09 |
| emily | but I expect taking the last value is by far the most common | 10:14:19 |
| emily | Python matches nlohmann here for instance | 10:14:30 |
| KFears& 🏳️⚧️ (they/them) | As in, if we would like to be strict and reject JSON with duplicate keys, we would also like to have an easily available option to parse it while choosing last key's value. Which works in general PL context, but is a large headache for embedded DSLs like NixLang | 10:14:40 |
 Qyriad | We've seen some abuses of duplicate keys to hack "comments" into JSON, lmao | 10:16:02 |
| KFears& 🏳️⚧️ (they/them) | We also feel like "last value overrides" is more intuitive of the "accept" options, because it matches the behavior of "set" operations on a hashmap and makes sense for top-to-bottom reading, while "first value overrides" feels not very programmer-ish, and "modify both keys to be unique" is very unexpected and footgunny | 10:17:05 |
| KFears& 🏳️⚧️ (they/them) | In reply to @qyriad:katesiria.org
We've seen some abuses of duplicate keys to hack "comments" into JSON, lmao This is horrifying | 10:17:30 |
| emily | actually the only ones that did are ones that crashed, lol | 10:18:11 |
| emily | oh wait no | 10:18:20 |
| emily | that was just a bad choice of colours | 10:18:28 |
| emily | anyway I don't think something called fromJSON should reject valid JSON unless there's truly no reasonable behaviour it could do with it | 10:19:01 |
| Qyriad | We agree | 10:19:14 |
| Qyriad | Python json.loads and jq both accept duplicate keys, with the last one winning | 10:21:25 |
 piegames | lol serde_json can't even detect duplicates | 10:27:05 |
| emily | isn't the serde model fundamentally event-based? I'd expect you can write a deserializer that detects them? | 10:27:53 |
| piegames | https://github.com/serde-rs/json/issues/1074 but also, this is the main reason why I hate "take the last value". It's bogus semantics, and on any input with duplicate fields the chances that it was generated in mistake is high. Taking the last value will lead to silent failure in such cases | 10:28:28 |
| piegames | it is possible with serde, and there is a PR, but currently serde_json offers no way to detect it | 10:28:54 |
 Coca | simd-json (the rust crate) seems to ignore duplication but keep the first one instead | 10:31:30 |
| Coca | nanoserde keeps the last one | 10:31:54 |
| * piegames sighs | 10:33:59 |
| piegames | I hate JSON so fucking much | 10:34:03 |
| piegames | currently reading the I-JSON spec and got painfully reminded that JSON has zero non-text handling capabilities | 10:34:37 |
 Sergei Zimmerman (xokdvium) | I recall there seemed to be some AWS endpoint that legitimately used duplicate keys too | 10:34:41 |
| Sergei Zimmerman (xokdvium) | Though I’m not sure I can find the reference now | 10:35:15 |
| piegames | let me rephrase the question, are there any reasonable use cases for duplicate keys where parsing both and then using only one of the values is the correct behavior? | 10:36:19 |
| emily | it's sorta like asking if there's any use case for parsing web pages with invalid HTML the same way everyone else parses them | 10:37:24 |
