| 2 Mar 2024 |
 delroth | there's also some potential value in the foundation not having to manage assets, as opposed to operational costs | 22:22:39 |
 raitobezarius | ah fun fact btw https://lists.debian.org/debian-snapshot/2024/02/msg00003.html | 22:23:22 |
| raitobezarius | olasd told me "this is what happens when you have 17 architectures used by 3 persons" when I pinged him about that hexa :D | 22:23:54 |
| delroth | also copying what I was saying on the #dev channel to make sure we have everything in one history:
we've had discussions about this in the past and came up to roughly the same cost estimates, the main issue is the big mindset change in having the current set of infra volunteers be in charge of the reliability of fairly complex infra directly in the main user query path. As much as I hate S3, nobody here has to be oncall for when it's down :)
(it doesn't disqualify a self-hosting solution, but uh, it's hard to have proper cost estimates that don't include a potential "we need to pay someone to be fulltime oncall")
| 22:24:45 |
| delroth | * also copying what I was saying on the #dev channel to make sure we have everything in one history:
we've had discussions about this in the past and came up to roughly the same cost estimates, the main issue is the big mindset change in having the current set of infra volunteers be in charge of the reliability of fairly complex infra directly in the main user query path. As much as I hate S3, nobody here has to be oncall for when it's down :)
(it doesn't disqualify a self-hosting solution, but uh, it's hard to have proper cost estimates that don't include a potential "we need to pay someone to be fulltime oncall")
| 22:24:49 |
| delroth | * also copying what I was saying on the #dev channel to make sure we have everything in one history:
we've had discussions about this in the past and came up to roughly the same cost estimates, the main issue is the big mindset change in having the current set of infra volunteers be in charge of the reliability of fairly complex infra directly in the main user query path. As much as I hate S3, nobody here has to be oncall for when it's down :)
(it doesn't disqualify a self-hosting solution, but uh, it's hard to have proper cost estimates that don't include a potential "we need to pay someone to be fulltime oncall")
| 22:25:00 |
| delroth | (AFAIK nobody has made a proper call on what kind of availability target we'd like to hit, so it's hard to know what kind of HA requirements as well as staffing we'd need) | 22:26:48 |
 hexa | to be fair, I'd expect nobody to know | 22:27:05 |
| raitobezarius | Arguably, I think the hard metric is the durability one | 22:27:05 |
| raitobezarius | Availability is one that matters but with a CDN in front, a lot of stuff can be mitigated | 22:27:21 |
| raitobezarius | And during us-east-1 outages, I don't think there was much to be noticed | 22:27:30 |
| delroth | I still think that if we run a "hot" / "recent" cache on Hetzner while keeping all the historical stuff on AWS, we can likely decrease the bill by a lot | 22:28:56 |
| raitobezarius | In reply to @hexa:lossy.network
to be fair, I'd expect nobody to know (it seems a political decision too tbh) | 22:29:27 |
| raitobezarius | (how many MB/year are you OK to lose?) | 22:29:34 |
| raitobezarius | * ("how many MB/year are you OK to lose?") | 22:29:37 |
| hexa | really depends on which MBs you are going to loose 😛 | 22:32:02 |
| raitobezarius | <insert meme about the dog "no choose; only lose"> | 22:32:17 |
 nh2 | In reply to @delroth:delroth.net
I still think that if we run a "hot" / "recent" cache on Hetzner while keeping all the historical stuff on AWS, we can likely decrease the bill by a lot I don't understand that; the cost on AWS is the historical stuff, because the cost per TB is high on AWS.
The other way around the argument would make sense | 22:34:31 |
| delroth | the cost on AWS is in large part bandwidth | 22:34:45 |
| raitobezarius | (80TB/month) | 22:34:51 |
| delroth | I don't have the exact breakdown, but $thousands/month | 22:35:14 |
| raitobezarius | I think it's 3K-ish | 22:35:22 |
| raitobezarius | OTOH | 22:35:26 |
| delroth | I thought it was more | 22:35:29 |
| raitobezarius | maybe I'm wrong, I don't have access to cost explorer | 22:35:38 |
| raitobezarius | I'm playing all those values by mind | 22:35:43 |
| raitobezarius | but then I think we can say ≥ 3K | 22:35:49 |
| delroth | In reply to @raitobezarius:matrix.org
maybe I'm wrong, I don't have access to cost explorer I have access to cost explorer but I don't think I know how to interpret it - I'm pretty sure the $9K/month discount is applied weirdly in there | 22:37:34 |
| raitobezarius | mmm, I remember we were able to isolate the bandwidth costs without the 9K/mo there but yeah | 22:37:57 |
| raitobezarius | during a meeting with zimbatm with the S3 folks | 22:38:06 |
