| 12 Feb 2025 |
 hexa | hrm 😕 | 20:37:02 |
 John Ericson | it does seem to have something to do wit the parallelism and the eval failure? | 20:37:35 |
| John Ericson | * it does seem to have something to do with the parallelism and the eval failure? | 20:37:45 |
| John Ericson | Should we have different pipes for different nix eval jobs workers? | 20:37:57 |
 emily | do you have multiple processes writing JSON to the same pipe? | 20:43:14 |
| emily | that seems like a recipe for disaster | 20:43:17 |
| hexa | yeah, could be that with low parallelisation they evade each other by sheer luck | 20:43:45 |
| hexa | but we have 16 workers right now | 20:43:48 |
| hexa | and I might want to increase that even further 😄 | 20:43:55 |
 K900 | Wait is there no sync on the output | 20:46:24 |
| 13 Feb 2025 |
| John Ericson | Mic92: can we has moar pipe? | 01:01:47 |
| John Ericson | we could do a unix domain socket | 01:01:52 |
| John Ericson | and then every worker connects afresh | 01:02:01 |
 Mic92 | What does a long line look like? | 01:10:50 |
| Mic92 | Don't understand yet what the issue is. Is perl reading not full lines? | 01:13:08 |
| hexa | it looked like multiple workers puking into the same pipe | 01:13:52 |
| hexa | which sometimes lead to json … mixed with other json | 01:14:11 |
| hexa | not sure if John Ericson knows anything else by now | 01:14:34 |
| Mic92 | Okay will check for today | 01:15:01 |
| Mic92 | I have machines with a lot of ram and core count but haven't seen that so far | 01:16:19 |
| Mic92 | But maybe libc needs to flush if it gets to the pipe buffer limit regardless of a missing newline | 01:17:32 |
| emily | I wouldn't rely on multiple processes writing into the same stream pipe. | 01:18:04 |
| emily | that's just the wrong primitive and will never work fully reliably | 01:18:17 |
| Mic92 | I can just add a lock around it | 01:18:27 |
| Mic92 | We are not really limited by the speed to write to stdout | 01:18:48 |
| Mic92 | On nixpkgs it's slow enough that I can almost read it | 01:19:06 |
| John Ericson | Mic92: yeah everything is in the issue | 01:21:22 |
| John Ericson | I know nothing that anyone else doesn't | 01:21:28 |
| John Ericson | I do agree with emily that multiple pipes/sockets feels better than locking | 01:21:52 |
| John Ericson | or are you sending everything back to the main process anyways, so it is in a good position to mux them? | 01:22:11 |
