| 19 Mar 2026 |
 hexa (signing key rotation when) | Works for me | 12:08:15 |
 Sergei Zimmerman (xokdvium) | Don't think so. With a dummy python server with h1.1 I do get reuse:
curl: Reusing existing http: connection with host localhost
downloading 'http://localhost:9000/9x7dq2sgrw63d93pa5lyk51hgwsmmn9k.narinfo'...
| 12:15:58 |
 Mic92 | https://github.com/NixOS/infra/pull/982 | 12:17:11 |
| Mic92 | I am checking also what the aws sdk is actually doing, since it's also using curl | 12:22:47 |
| Mic92 | https://github.com/aws/aws-sdk-cpp/blob/9204e236faaa1ca6a0342dee7caf61c7cf5ad8bb/src/aws-cpp-sdk-core/source/http/curl/CurlHandleContainer.cpp#L172-L176 | 12:24:33 |
| Mic92 | So looks like we always had keep-alive | 12:25:12 |
| Mic92 | but aws does it's own pooling | 12:26:48 |
| Sergei Zimmerman (xokdvium) | It might have retries for the error. Also one thing to note is that old code didn't run concurrent s3 requests at all, since it was using the blocking API. Now we fire off a bunch of requests in parallel. | 12:26:53 |
| Sergei Zimmerman (xokdvium) | We do curl_multi pooling too, that does reuse the handles | 12:27:25 |
| Sergei Zimmerman (xokdvium) | Or rather the connections for the easy handles | 12:27:36 |
| Mic92 | aws, doesn't seem to use this interface | 12:28:30 |
| Sergei Zimmerman (xokdvium) | Well it effectively does the same thing. What aws probably does is retry on that 400 error | 12:29:30 |
 Arian | In reply to @joerg:thalheim.io
but aws does it's own pooling Only if you use the transfer API iirc which we didn't | 12:32:54 |
| Arian | For S3 you also need to retry on 503 for uploads
https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance-design-patterns.html | 12:34:08 |
| Arian | They use 503 for rate limit :') | 12:34:30 |
| Mic92 | https://github.com/aws/aws-sdk-cpp/blob/9204e236faaa1ca6a0342dee7caf61c7cf5ad8bb/src/aws-cpp-sdk-core/source/client/CoreErrors.cpp#L90-L91 | 12:35:08 |
| Sergei Zimmerman (xokdvium) | Yup bernardo added that in https://github.com/NixOS/nix/pull/15449 | 12:35:11 |
| Arian | Sweet | 12:35:38 |
| Mic92 | where dos this handle 400? | 12:37:33 |
| Mic92 | * where does this handle 400? | 12:37:39 |
| Sergei Zimmerman (xokdvium) | I was talking about 503, 400 is not handled there. We'd need to look at the response xml | 12:38:17 |
| Arian | Line 113 | 12:38:56 |
| Arian | Decides if an error is retry able | 12:39:11 |
| Mic92 | True, but I think Sergei is right, we need to look at the xml | 12:41:00 |
| Mic92 | deploying this now | 12:44:59 |
| Mic92 | how did you came up with this number, so I can monitor? | 12:49:53 |
| hexa (signing key rotation when) | journalctl -u hydra-queue-runner --since "2 days ago" --grep="unable to upload" | wc -l | 12:50:38 |
| Mic92 | okay, we don't have any upload error since: Thu Mar 19 01:01:45 PM UTC 2026 | 13:02:00 |
| Mic92 | Let's see | 13:02:05 |
| Mic92 | journalctl -u hydra-queue-runner --since "3 hour ago" --grep="unable to upload" is also empty | 13:02:13 |
