| 8 May 2026 |
 emily | I forget what the conclusion of that discussion was (I know the focus was on missing paths because of the access pattern but presumably those are not what's causing these expenses) | 11:11:26 |
| emily | I guess the issue is that if it's sufficiently spread out/high cardinality no per-path caching will help.
(though it seems surprising for scrapers to be going out of their way to find old stuff to query, I really doubt N versions of the same binaries are valuable?)
| 11:12:57 |
 Arian | Missing paths don't generate bandwidth cost. They generate API call cost. Which is small | 11:33:00 |
| emily | right | 11:58:09 |
| emily | but I mean if a present path is being hammered, how long does Fastly cache that before going back to S3? | 11:58:33 |
| Arian | 24h i think | 11:58:46 |
| emily | IIRC you said it was not that long? | 11:58:53 |
| emily | yeah | 11:58:57 |
| emily | so I wonder if just increasing that a ton would help? | 11:59:10 |
| emily | I don't know how much Fastly will cache before evicting things though. but at least there's definitely no reason to evict something just because it's been a day :) | 11:59:59 |
 hexa | do you think we can get better caching than what fastly currently provides? | 13:13:50 |
| emily | (not sure if you're asking me but) if it expires every 24 hours then a bot that hits a bunch of store paths every 24 hours and then repeats causes costs every day vs. potentially getting cached indefinitely if we tell Fastly there's no need to expire known store paths right?
(but obviously it's just throwing things at the wall unless it's known what the access pattern looks like. still I imagine it's good in general for e.g. the latest stable installer ISO to not get redownloaded from S3 every day?)
| 13:17:15 |
| hexa | 
Download | 13:19:25 |
| hexa | 
Download | 13:20:08 |
| emily | but it's precisely that 5% that must be causing ^ right? 🤔 | 13:20:50 |
| hexa | at the same time | 13:21:00 |
| hexa | 
Download | 13:21:02 |
| hexa | 
Download | 13:21:22 |
| emily | Nix probably counts as "other bots"? | 13:21:57 |
| emily | 0 DDoS requests mitigated is a fun figure | 13:22:14 |
| hexa | I would imagine it does, since it doesn't advertise as a browser | 13:26:04 |
| 9 May 2026 |
| hexa | Arian what's blocking https://github.com/NixOS/infra/pull/728 | 12:17:45 |
| Arian | ADHD mostly | 12:24:01 |
| Arian | I keep forgetting i was doing this | 12:24:09 |
| Arian | Only thing I'm a bit afraid of is that if we actually have people scraping old paths intelligent tiering might actually be more expensive because things get moved into more expensive tiers... But idk I think we should just apply and observe for a month. | 13:48:08 |
| Arian | Worst case is we revert | 13:48:13 |
| hexa | yeah, we can't know without trying | 13:55:45 |
| hexa | if things go to shit, what's next? gc? | 13:55:59 |
 lassulus | we do the cache exfil anyway? so maybe gc? or we get more free credits from aws. but there are multiple ideas floating around what happens if the egress costs eat up the free credits we get from amazon. would not worry too much about it for now | 13:58:00 |
| lassulus | I opushed https://github.com/NixOS/infra/pull/728 I would be happy to deploy it, but not sure if I have the right credentials :D | 13:58:29 |
