| 10 Jul 2025 |
 Arian | yes | 18:59:37 |
 emily | (…does S3 bill for 404s?) | 18:59:39 |
| Arian | Yes S3 bills for 404s | 18:59:46 |
| emily | lol | 18:59:49 |
| Arian | they even used to bill for authorization errors so you could just rack up anyone's bill by knowing their bucket name | 19:00:02 |
| Arian | they changed that now | 19:00:04 |
| emily | please run some numbers on how much of the cache size bill this is, I'm so curious | 19:00:13 |
 Jeremy Fleischman (jfly) | no, but the intent is to "only" cache 404s for 24 hours | 19:00:14 |
| Arian | I have these numbers. I don't think API Calls are a large portion of our cost | 19:00:36 |
| emily | that might not be great UX: the channel scripts run to bump channels after the final builds complete | 19:00:41 |
| emily | so anyone who has been running master before that will have cached 404s for first day everyone is bumping to the new channel revision | 19:00:58 |
 Zhaofeng Li | contributors can try to build locally and trigger the negative caching, which would be bad UX | 19:01:07 |
| Zhaofeng Li | yeah | 19:01:10 |
| Arian | But this is already a problem. Nix caches 404s locally | 19:01:18 |
| emily | it's not a problem across users | 19:01:25 |
| Zhaofeng Li | but this would affect it for everyone | 19:01:29 |
| emily | most users don't try to run master, but some do | 19:01:30 |
| Arian | Aaah I see | 19:01:32 |
| Zhaofeng Li | some Nix hater can even just look at the big in-progress hydra builds, and hit cache.nixos.org with the outpaths | 19:02:18 |
| emily | so even someone testing a build before merging could screw up its caching for the next channel bump | 19:02:19 |
| emily | so definitely needs cache-busting or to be much shorter than 24h I think | 19:02:31 |
| Zhaofeng Li | * some Nix hater can even just look at the big in-progress hydra builds, and hit cache.nixos.org with the outpaths... then laptops everywhere will be building chromium | 19:03:01 |
| Arian | if I'm reading the bill right we currently spend 570 dollars per month on GET requests and only 60 dollars per month on PUT
PUT is charged | 19:08:55 |
| Arian | * if I'm reading the bill right we currently spend 570 dollars per month on GET requests and only 60 dollars per month on PUT
| 19:09:10 |
| Arian | okay so seems not caching 404s is kind of a "feature" | 19:14:14 |
| Arian | because it means we don't need to do cache-busting on channel bumps | 19:14:39 |
| Arian | but it feels kinda annoying that the most common request is the slowest request | 19:14:50 |
| Arian | Found old meeting notes:
https://nixos.wiki/wiki/Maintainers:Fastly
| 19:15:48 |
| Arian |
Cache aggressive: ~1 month. (STATUS: NOT DONE -- requires upstream Hydra tooling changes, so cache uploads have their potential 404s purged in a timely manner.)
| 19:15:57 |
| emily | I would guess sending a purge request would not be a very invasive Hydra change. (maybe if they need to be batched?) | 19:19:56 |
