!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

391 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.118 Servers

You have reached the beginning of time (for this room).


SenderMessageTime
19 Nov 2025
@vcunat:matrix.orgvcunatBut you can see it in the log above.16:05:08
@vcunat:matrix.orgvcunatThey used to be 200, now they're 302.16:05:17
@adam:robins.wtf@adam:robins.wtfI can do some digging on the redirects16:07:55
@adam:robins.wtf@adam:robins.wtfAre there log events from before this stopped working?16:08:09
@adam:robins.wtf@adam:robins.wtfIt looks like redirects should be followed if I'm reading the go docs correctly. https://pkg.go.dev/net/http#Client.Do and I'd expect a different error if the download itself failed https://github.com/lxc/incus/blob/09e0f36dd5f02aeda5c97a4b27060cef338e1d76/shared/util/net.go#L5216:14:25
@adam:robins.wtf@adam:robins.wtf* It looks like redirects should be followed if I'm reading the go docs correctly. https://pkg.go.dev/net/http#Client.Do and I'd expect a different error if the download itself failed (aka not 200) https://github.com/lxc/incus/blob/09e0f36dd5f02aeda5c97a4b27060cef338e1d76/shared/util/net.go#L5216:14:50
@adam:robins.wtf@adam:robins.wtfyes this is all too convoluted, and i have some ideas on how to avoid distrobuilder for this but they will take more time and convincing upstream to accept them. unfortunately if we don't fix this before 10 days after the failure started our images will expire and users won't be able to pull them down from the LXC image server.16:18:45
@adam:robins.wtf@adam:robins.wtfi'll dig into distrobuilder some more to see if i can at the result it's failing to unpack16:28:28
@vcunat:matrix.orgvcunatBased on the logs, the "distrobuilder" agent does not follow the redirect.16:28:37
@vcunat:matrix.orgvcunatOh wait.16:29:08
@vcunat:matrix.orgvcunatWrong command at first, but correct statement, I think.16:31:53
@vcunat:matrix.orgvcunat
grep -F download-by-type/file/squashfs-image /var/log/nginx/access.log | grep -F distrobuilder | grep -F /build/

This command shows empty. Without the last grep I see them. i.e. distrobuild agents only look at the /job/nixos/... URLs and not the followup /build/... URLs.

16:33:11
@adam:robins.wtf@adam:robins.wtf👍 thanks16:34:02
@adam:robins.wtf@adam:robins.wtfIt's anubis16:54:26
@adam:robins.wtf@adam:robins.wtfwhich also explains why it's not following redirects. it gets the anubis page returned and doesn't even know about them16:55:39
@adam:robins.wtf@adam:robins.wtf
<p>This website is running Anubis version <code>v1.23.1</code>.</p>
16:57:43
@vcunat:matrix.orgvcunatOK, so you need an exception to bypass Anubis for this use case.17:06:33
@vcunat:matrix.orgvcunat adamcstephens: can you retry now? 17:10:24
@vcunat:matrix.orgvcunat(just trying a quick prototype)17:10:43

Show newer messages


Back to Room ListRoom Version: 6