| 1 Jun 2025 |
 edef | we can fetch into s3 on the aws side, pulling from fastly | 11:50:12 |
| edef | whether that is wise is another question, but we can definitely do it | 11:50:27 |
| edef | mostly it is isomorphic to having hydra have a more-local / free-egress storage service | 11:51:49 |
| edef | and using s3 mostly as archival storage | 11:52:05 |
 toonn | That's certainly better than Hydra pushing to S3 and Fastly pulling from it preferentially but won't most of the Fastly cache still be clobbered by the S3 requests? So in the end Fastly would fetch many things from Hydra twice still? | 12:29:43 |
 flokli | We can also do the push through thing. So instead of hydra uploading to fastly or s3 directly, it'd send to a http daemon that uploads to both places | 13:47:44 |
| flokli | It gives nicer control over the import path | 13:48:01 |
| edef | yes, these are all essentially identical in terms of dataflow | 13:48:06 |
| flokli | * It gives nicer control over the upload path | 13:48:12 |
| edef | * yes, these are all essentially identical in terms of bulk data flow | 13:48:20 |
| edef | or well, we would be spending twice the hetzner bandwidth if we go to both | 13:48:57 |
| edef | mostly it depends on how much annoying orchestration we want to have | 13:49:16 |
| edef | * mostly it depends on how much annoying orchestration and consistency trickery we want to have | 13:49:29 |
| flokli | I assumed network bandwidth egressing from Hetzner is not the bottleneck | 13:51:01 |
 hexa | it is not | 13:51:12 |
| edef | it's not, no | 13:51:15 |
| edef | so like, for the most part you can do anything to the NAR datapath | 13:52:06 |
| edef | as long as narinfo 404 is fast | 13:52:16 |
|  BMG joined the room. | 15:29:29 |
 Arian | In reply to @flokli:matrix.org
We can also do the push through thing. So instead of hydra uploading to fastly or s3 directly, it'd send to a http daemon that uploads to both places I'm suggesting something way simpler.
We just have nix-serve running on hydra and serve its nix store.
We configure fastly to first fetch Hydra and on 404 fetch s3 | 17:41:24 |
| Arian | Requires no big changes on hydra side | 17:41:36 |
| edef | the narinfo 404 path is sacred, we cause a lot of user suffering if it isn't fast | 17:42:18 |
| Arian | But it might be too simplistic. | 17:42:18 |
| edef | whether this was a good design choice in the substitution mechanism is another matter | 17:42:40 |
| edef | but this is the army we fight with, for now | 17:42:55 |
| Arian | But it will just be slow for a single person. Afterwards it's cached | 17:43:02 |
| edef | no | 17:43:08 |
| Arian | :D | 17:43:09 |
| edef | 404s aren't cacheable, since presence isn't immutable | 17:43:23 |
| Arian | Aaaaaah | 17:43:33 |
