| 18 Jun 2026 |
 Arian | In reply to @joerg:thalheim.io
You can't really stream zstd with s3 unfortunally because s3 wants to know the size up-front. In niks3 I solved this by having smallish multi-part upload. I just yesterday introduced something similar in queue-runner because before it was just buffering everything in memory. S3 doesn't require this | 16:14:18 |
| Arian | Or maybe I'm misunderstanding what we mean with size upfront | 16:14:49 |
 Sergei Zimmerman (xokdvium) | I mean, if we want to be able to retry/send the hashsums as x-amz- headers and stuff yeah, it’s kinda necessary to keep the file around.
In nix we that by compressing to an O_TMPFILE fd and seeking in it when we need to retry | 16:17:10 |
 Mic92 | AWS maybe, but there are also other s3 compatible endpoints that require 'Content-Length' headers to be set. | 16:35:12 |
| Arian | Ah wait maybe you're right. NVM | 17:33:43 |
| Arian | Except for multipart uploads but we don't need those for small objects | 17:33:59 |
| Sergei Zimmerman (xokdvium) | In reply to @arianvp:matrix.org
Ah wait maybe you're right. NVM I don’t think we set Content-Length in nix’s s3 client? | 17:35:01 |
| Sergei Zimmerman (xokdvium) | Just for POST, not for PUT | 17:35:37 |
| Sergei Zimmerman (xokdvium) | Ah no, sorry. We did in fact set it for both | 17:36:22 |
 hexa | https://grafana.nixos.org/d/hejbnzw/iocaine | 17:44:30 |
| hexa | hydra feels much more responsive when it is not hammered by bots | 17:48:09 |
| hexa | that manage to bypass anubis | 17:48:14 |
 magic_rb | Did you swap anubis with iocaine? | 17:48:43 |
| hexa | looks like unstable-small just advanced | 18:04:48 |
|  whispers [& it/fae] changed their display name from whispers [& it/fae] to meow meow. | 18:46:02 |
| whispers [& it/fae] changed their display name from meow meow to whispers [& it/fae]. | 19:11:49 |
|  stv0g joined the room. | 19:14:02 |
|  Grayson Tinker joined the room. | 19:31:49 |
| Mic92 | Looks I got it fixed now with my parallel upload code: https://hydra.nixos.org/machines all machines are building and rarely upload | 19:59:15 |
| Mic92 | * Looks I got it fixed now with the new parallel upload code: https://hydra.nixos.org/machines all machines are building and rarely upload | 19:59:22 |
| Mic92 | * Looks I got it fixed now with the new parallel upload code: https://hydra.nixos.org/machines all machines are building and rarely uploading | 19:59:35 |
|  Tobi joined the room. | 20:25:50 |
| 19 Jun 2026 |
 Vladimír Čunát | Same derivation is being built on two machines at once? At a glance, this happens even in a situation where the previous queue-runner wouldn't do it:
https://hydra.nixos.org/build/331659169#tabs-buildsteps | 03:28:10 |
| Mic92 | Did the first build actually finish? | 05:31:00 |
| Mic92 | Maybe just a missing DB update | 05:31:27 |
 K900 | https://hydra.nixos.org/build/331675357 another funni | 05:38:13 |
| Mic92 | I just checked just looks like stale db entries | 05:48:11 |
| Mic92 | There are not actually two builds running | 05:48:27 |
| Mic92 | Looks the db isn't properly updated | 05:49:16 |
| Vladimír Čunát | The first one did not run until the end. | 05:49:20 |
