| 2 Mar 2024 |
 raitobezarius | (plus, discussing with Hetzner to do the operations) | 01:47:01 |
| raitobezarius | (BTW, infra people just removed NixOps — finally — :D — but I guess anything modern as deployment framework can work) | 01:47:28 |
 nh2 | raitobezarius: what is the latest discourse post that shows the current spending on hosting and how much of that AWS gives us for free? | 01:47:38 |
| raitobezarius | In reply to @nh2:matrix.org
raitobezarius: what is the latest discourse post that shows the current spending on hosting and how much of that AWS gives us for free? cache is 15K USD/mo | 01:47:51 |
| raitobezarius | 9K of it is free credit from AWS | 01:47:57 |
| raitobezarius | 6K of it is out of our own pocket | 01:48:02 |
| nh2 | So I guess "the Foundation doesn't have 1470EUR/mo" is referring to "... on top of the AWS spending, vs instead of it"? | 01:48:48 |
| raitobezarius | I'd need to jump again in the finances to look at all of that | 01:49:37 |
| raitobezarius | but I don't think we can support it on the long term that easily | 01:49:44 |
| raitobezarius | even out of AWS spending | 01:49:50 |
| raitobezarius | the big problem also is that running the operations to have a self hosted cache is currently difficult | 01:50:02 |
| raitobezarius | so it would have to be on the top of AWS spending to avoid disasters | 01:50:10 |
| raitobezarius | there's a double problem that is | 01:50:32 |
| raitobezarius | (a) AWS offers a "nice" durability metric that cannot be reproduced with a Hetzner setup | 01:50:45 |
| raitobezarius | (b) Infra people may not have the bandwidth to touch this in the next months | 01:50:55 |
| raitobezarius | (c) Foundation may not have the money/energy, when it will come to it, to go for that solution | 01:51:11 |
| raitobezarius | (actually a triple problem) | 01:51:16 |
| raitobezarius | Another solution I am personally pursuing is just to get 1PB myself and put it in a non-super-available colo but almost free colo and duplicate the data | 01:52:46 |
| nh2 | I mean for the 5k/month difference you can literally hire Ceph experts full time to run the cluster for you | 01:54:25 |
| raitobezarius | yeah but 6K/mo not being sustainable means that even by throwing 5k/mo to hire Ceph experts full time, this won't be a viable solution, no? | 01:54:48 |
| raitobezarius | Right now, the trajectory the Foundation is taking is to launch garbage collection to reduce those 6K/mo | 01:55:19 |
| nh2 | In reply to @raitobezarius:matrix.org
yeah but 6K/mo not being sustainable means that even by throwing 5k/mo to hire Ceph experts full time, this won't be a viable solution, no? Until the data grows another 2 years, when it's 10k/mo | 01:55:23 |
| raitobezarius | In reply to @nh2:matrix.org
Until the data grows another 2 years, when it's 10k/mo Well, given the current growth pattern, it won't need 2 years | 01:55:41 |
| raitobezarius | Either way, it seems there's a dire need of | 01:56:02 |
| raitobezarius | (a) a long term solution that is cost-efficient and low maintenance for the community (or somehow we find a way to pay people to work on infra), e.g. ≤ N€/mo | 01:56:35 |
| raitobezarius | (b) a very short term solution to avoid dropping 50K USD to AWS just for storing 500TB and moving 80TB/mo | 01:56:54 |
| raitobezarius | (c) a way to clarify what is the budget for the binary cache that is being operated and a way to secure that money | 01:57:20 |
| nh2 | In reply to @raitobezarius:matrix.org
Another solution I am personally pursuing is just to get 1PB myself and put it in a non-super-available colo but almost free colo and duplicate the data I guess this might also be an approach, just launch a "competing" binary store, that can have the benefits of not needing GC because it's cheaper, and perhaps also faster.
For colo you want to buy the hardware? Is it also in Hetzner colo? | 01:57:33 |
| raitobezarius | It's not in Hetzner | 01:57:45 |
| raitobezarius | I'm based in France | 01:58:10 |
