| 19 Nov 2025 |
 adamcstephens | yes this is all too convoluted, and i have some ideas on how to avoid distrobuilder for this but they will take more time and convincing upstream to accept them. unfortunately if we don't fix this before 10 days after the failure started our images will expire and users won't be able to pull them down from the LXC image server. | 16:18:45 |
| adamcstephens | i'll dig into distrobuilder some more to see if i can at the result it's failing to unpack | 16:28:28 |
 Vladimír Čunát | Based on the logs, the "distrobuilder" agent does not follow the redirect. | 16:28:37 |
| Vladimír Čunát | Oh wait. | 16:29:08 |
| Vladimír Čunát | Wrong command at first, but correct statement, I think. | 16:31:53 |
| Vladimír Čunát | grep -F download-by-type/file/squashfs-image /var/log/nginx/access.log | grep -F distrobuilder | grep -F /build/
This command shows empty. Without the last grep I see them. i.e. distrobuild agents only look at the /job/nixos/... URLs and not the followup /build/... URLs.
| 16:33:11 |
| adamcstephens | 👍 thanks | 16:34:02 |
| adamcstephens | It's anubis | 16:54:26 |
| adamcstephens | which also explains why it's not following redirects. it gets the anubis page returned and doesn't even know about them | 16:55:39 |
| adamcstephens | <p>This website is running Anubis version <code>v1.23.1</code>.</p>
| 16:57:43 |
| Vladimír Čunát | OK, so you need an exception to bypass Anubis for this use case. | 17:06:33 |
| Vladimír Čunát | adamcstephens: can you retry now? | 17:10:24 |
| Vladimír Čunát | (just trying a quick prototype) | 17:10:43 |
| adamcstephens | still got the anubis page | 17:11:14 |
| Vladimír Čunát | I suppose someone more capable than me in this would have a look 😅 | 17:17:20 |
| adamcstephens | In theory a bot policy as below should work, but it isn't scoped at all if that is desired. I don't see managed policies already in nixos/infra though, and not sure how the defaults are handled by adding one...
- name: distrobuilder
user_agent_regex: distrobuilder
action: ALLOW
| 17:21:57 |
| Vladimír Čunát | I thought we have these here
https://github.com/NixOS/infra/blob/main/build/hydra-proxy.nix#L96 | 17:23:15 |
| Vladimír Čunát | But maybe I'm wrong. | 17:23:24 |
| Vladimír Čunát | i.e. the approach is to allow some URLs, not some agents. | 17:24:06 |
| adamcstephens | ahh | 17:24:59 |
 hexa | I assume an anubis update was backported and that caused this change in behavior | 17:31:19 |
| adamcstephens | that rule should handle the intermediate and final redirects, but not the initial request /job/nixos/trunk-combined/nixos.incusContainerImage.x86_64-linux/latest/download-by-type/file/squashfs-image | 17:31:27 |
| adamcstephens | yeah, i kinda guessed it was an anubis update given the infra git history | 17:31:43 |
| adamcstephens | how about /job/[^/]+/[^/]+/[^/]+/latest/download-by-type/[^/]+ ? | 17:39:47 |
| adamcstephens | maybe a bit more explicit than the others, so could drop the last match | 17:42:04 |
| Vladimír Čunát | 🤷 deployed
locations."~ ^/job/[^/]+/[^/]+/[^/]+/latest/download-by-type/[^/]+" = {
proxyPass = "http://hydra-server";
};
(temporarily)
| 17:42:55 |
| adamcstephens | hmm, still no luck | 17:44:24 |
| adamcstephens | strangely it works for curl with a distrobuilder user-agent, but not with the go program itself. | 17:48:50 |
| adamcstephens | I'm assuming that's anubis trusting curl, but detecting the go application and blocking it | 17:50:48 |
| adamcstephens | not sure why that regex isn't matching though. :/ | 17:58:33 |
