!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

418 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.128 Servers

You have reached the beginning of time (for this room).


SenderMessageTime
8 May 2026
@arianvp:matrix.orgArianThat sounds really off to me. 10:10:17
@arianvp:matrix.orgArian1000065300.jpg
Download 1000065300.jpg
10:11:44
@arianvp:matrix.orgArianRed line is egress bandwidth10:11:51
@arianvp:matrix.orgArianIt's hockey sticking 10:11:57
@jappie:jappie.devjappiedoes that mean that the amount of requests for stuff that isn't cached by fastly is growing (perhaps scrapers stumbling upon old derivations)? because I'd assume an uptick in users downloading new-ish derivations would mean more hits in fastly and no noticeable growth in S3 egress10:17:43
@hexa:lossy.networkhexa (signing key rotation when)I think so10:18:23
@jappie:jappie.devjappiehow the hell are scrapers discovering old store paths / derivations... the URLs for those contain a hash right? I'd expect that to be really difficult and time-consuming (and useless) to scrape10:19:37
@arianvp:matrix.orgArianYou can dump all of hydras evaluations. Or run evaluations for all historical nixos commits yourself10:21:14
@leona:leona.isleonacan we determine that this is egress via fastly or is someone downloading them directly from AWS?10:21:46
@arianvp:matrix.orgArianDirectly through S3 is only possible when requester pays10:22:01
@arianvp:matrix.orgArianWe don't have anonymous auth enabled on our bucket. You need to provide your iam identity and it gets billed to the caller10:22:32
@arianvp:matrix.orgArianIt would be preferable if scrapers would scrape S3 directly as then it doesn't cost us10:22:55
@hexa:lossy.networkhexa (signing key rotation when)one obvious fix would be to GC harder, provide fewer targets10:24:42
@arianvp:matrix.orgArianI'm wondering if I can somehow figure out from S3 the distribution of the age of objects being requested 10:30:52
@emilazy:matrix.orgemilyis it still true that Fastly doesn't cache paths for long even once built?11:10:55

Show newer messages


Back to Room ListRoom Version: 6