| 18 Jun 2026 |
 hexa | the old queue runner could do unstable-small in 30 minutes with fewest rebuilds | 15:37:13 |
| hexa | Redacted or Malformed Event | 15:37:21 |
| hexa | as a comparison | 15:37:39 |
 K900 | How good is the otel instrumentation currently on the runner/workers? | 15:38:57 |
| K900 | It might be good to spin up a Tempo instance so we can have proper metrics for this | 15:39:17 |
| K900 | Should be fairly easy to do | 15:39:27 |
 Mic92 | You can't really stream zstd with s3 unfortunally because s3 wants to know the size up-front. In niks3 I solved this by having smallish multi-part upload. I just yesterday introduced something similar in queue-runner because before it was just buffering everything in memory. | 15:40:14 |
| Mic92 | But I can try to make the parts a bit smaller. | 15:40:40 |
| hexa | I don't immediately see an option to set up an otel endpoint in the module | 15:46:17 |
| hexa | option('otel', type: 'feature', value: 'disabled', description: 'Enable OpenTelemetry support for Rust components')
| 15:46:50 |
| hexa | does it just eat OTEL_EXPORTER_OTLP_ENDPOINT? | 16:02:55 |
| hexa | Redacted or Malformed Event | 16:03:03 |
| Mic92 | I does but it also needs to be enabled in the package from the looks of things. | 16:07:04 |
| Mic92 | * Itdoes but it also needs to be enabled in the package from the looks of things. | 16:07:11 |
| Mic92 | * It does but it also needs to be enabled in the package from the looks of things. | 16:07:15 |
| hexa | yeah, this meson option | 16:07:27 |
 Arian | In reply to @joerg:thalheim.io
You can't really stream zstd with s3 unfortunally because s3 wants to know the size up-front. In niks3 I solved this by having smallish multi-part upload. I just yesterday introduced something similar in queue-runner because before it was just buffering everything in memory. S3 doesn't require this | 16:14:18 |
| Arian | Or maybe I'm misunderstanding what we mean with size upfront | 16:14:49 |
 Sergei Zimmerman (xokdvium) | I mean, if we want to be able to retry/send the hashsums as x-amz- headers and stuff yeah, it’s kinda necessary to keep the file around.
In nix we that by compressing to an O_TMPFILE fd and seeking in it when we need to retry | 16:17:10 |
| Mic92 | AWS maybe, but there are also other s3 compatible endpoints that require 'Content-Length' headers to be set. | 16:35:12 |
| Arian | Ah wait maybe you're right. NVM | 17:33:43 |
| Arian | Except for multipart uploads but we don't need those for small objects | 17:33:59 |
| Sergei Zimmerman (xokdvium) | In reply to @arianvp:matrix.org
Ah wait maybe you're right. NVM I don’t think we set Content-Length in nix’s s3 client? | 17:35:01 |
| Sergei Zimmerman (xokdvium) | Just for POST, not for PUT | 17:35:37 |
| Sergei Zimmerman (xokdvium) | Ah no, sorry. We did in fact set it for both | 17:36:22 |
| hexa | https://grafana.nixos.org/d/hejbnzw/iocaine | 17:44:30 |
| hexa | hydra feels much more responsive when it is not hammered by bots | 17:48:09 |
| hexa | that manage to bypass anubis | 17:48:14 |
 magic_rb | Did you swap anubis with iocaine? | 17:48:43 |
| hexa | looks like unstable-small just advanced | 18:04:48 |
