| 19 Nov 2025 |
 Vladimír Čunát | Oh wait. | 16:29:08 |
| Vladimír Čunát | Wrong command at first, but correct statement, I think. | 16:31:53 |
| Vladimír Čunát | grep -F download-by-type/file/squashfs-image /var/log/nginx/access.log | grep -F distrobuilder | grep -F /build/
This command shows empty. Without the last grep I see them. i.e. distrobuild agents only look at the /job/nixos/... URLs and not the followup /build/... URLs.
| 16:33:11 |
 adamcstephens | 👍 thanks | 16:34:02 |
| adamcstephens | It's anubis | 16:54:26 |
| adamcstephens | which also explains why it's not following redirects. it gets the anubis page returned and doesn't even know about them | 16:55:39 |
| adamcstephens | <p>This website is running Anubis version <code>v1.23.1</code>.</p>
| 16:57:43 |
| Vladimír Čunát | OK, so you need an exception to bypass Anubis for this use case. | 17:06:33 |
| Vladimír Čunát | adamcstephens: can you retry now? | 17:10:24 |
| Vladimír Čunát | (just trying a quick prototype) | 17:10:43 |
| adamcstephens | still got the anubis page | 17:11:14 |
| Vladimír Čunát | I suppose someone more capable than me in this would have a look 😅 | 17:17:20 |
| adamcstephens | In theory a bot policy as below should work, but it isn't scoped at all if that is desired. I don't see managed policies already in nixos/infra though, and not sure how the defaults are handled by adding one...
- name: distrobuilder
user_agent_regex: distrobuilder
action: ALLOW
| 17:21:57 |
| Vladimír Čunát | I thought we have these here
https://github.com/NixOS/infra/blob/main/build/hydra-proxy.nix#L96 | 17:23:15 |
| Vladimír Čunát | But maybe I'm wrong. | 17:23:24 |
| Vladimír Čunát | i.e. the approach is to allow some URLs, not some agents. | 17:24:06 |
| adamcstephens | ahh | 17:24:59 |
 hexa | I assume an anubis update was backported and that caused this change in behavior | 17:31:19 |
| adamcstephens | that rule should handle the intermediate and final redirects, but not the initial request /job/nixos/trunk-combined/nixos.incusContainerImage.x86_64-linux/latest/download-by-type/file/squashfs-image | 17:31:27 |
| adamcstephens | yeah, i kinda guessed it was an anubis update given the infra git history | 17:31:43 |
| adamcstephens | how about /job/[^/]+/[^/]+/[^/]+/latest/download-by-type/[^/]+ ? | 17:39:47 |
| adamcstephens | maybe a bit more explicit than the others, so could drop the last match | 17:42:04 |
| Vladimír Čunát | 🤷 deployed
locations."~ ^/job/[^/]+/[^/]+/[^/]+/latest/download-by-type/[^/]+" = {
proxyPass = "http://hydra-server";
};
(temporarily)
| 17:42:55 |
| adamcstephens | hmm, still no luck | 17:44:24 |
| adamcstephens | strangely it works for curl with a distrobuilder user-agent, but not with the go program itself. | 17:48:50 |
| adamcstephens | I'm assuming that's anubis trusting curl, but detecting the go application and blocking it | 17:50:48 |
| adamcstephens | not sure why that regex isn't matching though. :/ | 17:58:33 |
| Vladimír Čunát | 🤔 appended another /[^/]+ to cover the whole URL, I think. But I don't know. | 18:05:54 |
| adamcstephens | i think the ~ is a partial match. but i'm also pretty rusty with nginx rules | 18:10:55 |
| adamcstephens | still fails | 18:11:16 |
