| 2 Mar 2024 |
 raitobezarius | yeah but 6K/mo not being sustainable means that even by throwing 5k/mo to hire Ceph experts full time, this won't be a viable solution, no? | 01:54:48 |
| raitobezarius | Right now, the trajectory the Foundation is taking is to launch garbage collection to reduce those 6K/mo | 01:55:19 |
 nh2 | In reply to @raitobezarius:matrix.org
yeah but 6K/mo not being sustainable means that even by throwing 5k/mo to hire Ceph experts full time, this won't be a viable solution, no? Until the data grows another 2 years, when it's 10k/mo | 01:55:23 |
| raitobezarius | In reply to @nh2:matrix.org
Until the data grows another 2 years, when it's 10k/mo Well, given the current growth pattern, it won't need 2 years | 01:55:41 |
| raitobezarius | Either way, it seems there's a dire need of | 01:56:02 |
| raitobezarius | (a) a long term solution that is cost-efficient and low maintenance for the community (or somehow we find a way to pay people to work on infra), e.g. ≤ N€/mo | 01:56:35 |
| raitobezarius | (b) a very short term solution to avoid dropping 50K USD to AWS just for storing 500TB and moving 80TB/mo | 01:56:54 |
| raitobezarius | (c) a way to clarify what is the budget for the binary cache that is being operated and a way to secure that money | 01:57:20 |
| nh2 | In reply to @raitobezarius:matrix.org
Another solution I am personally pursuing is just to get 1PB myself and put it in a non-super-available colo but almost free colo and duplicate the data I guess this might also be an approach, just launch a "competing" binary store, that can have the benefits of not needing GC because it's cheaper, and perhaps also faster.
For colo you want to buy the hardware? Is it also in Hetzner colo? | 01:57:33 |
| raitobezarius | It's not in Hetzner | 01:57:45 |
| raitobezarius | I'm based in France | 01:58:10 |
| raitobezarius | I personally think buying the HW comes cheaper in the end | 01:58:30 |
| raitobezarius | But then we have to manage HW | 01:58:34 |
| raitobezarius | I already do it for a bunch of hardware, but it's not something that thousands of people depend on, so it's another discussion | 01:58:57 |
| nh2 | Theoretically it does not matter so much. You can buy the HW, ship it to hetzner, and pay them 21 EUR for 15 minutes "Remote hands"
https://docs.hetzner.com/robot/colocation/faq/#can-hetzner-online-install-my-hardware | 01:59:34 |
| raitobezarius | People may have other ideas like freezing the bucket in Glacier temporarily, then sending it to R2/Backblaze B2, or doing slow transfer proxy by replicating any data that gets requested via Fastly | 01:59:35 |
| raitobezarius | In reply to @nh2:matrix.org
Theoretically it does not matter so much. You can buy the HW, ship it to hetzner, and pay them 21 EUR for 15 minutes "Remote hands"
https://docs.hetzner.com/robot/colocation/faq/#can-hetzner-online-install-my-hardware Yeah, I guess | 02:00:24 |
| raitobezarius | I just don't know if we can afford or if we want to go for Hetzner Colo vs. Hetzner managed hardware if we go for a Hetzner service | 02:00:42 |
| raitobezarius | All of that is very fuzzy in my mind and dependent on current infrastructure team decisions / will | 02:00:54 |
| raitobezarius | So I would rather avoid pushing on them anything | 02:01:00 |
| raitobezarius | Also why I build my own solution on my side | 02:01:10 |
| raitobezarius | Overall, what I believe is that if someone comes in and drops 50K on the Foundation for the objective of duplicating the binary cache to save it, it's a solved problem | 02:01:46 |
| raitobezarius | Under that amount, you are kinda stuck having to ask the Foundation to consider taking the risk | 02:01:57 |
| raitobezarius | And it's not clear that it's a right choice or not | 02:02:04 |
| nh2 | For your colo, does it offer separate datacenters that isolated failure domains against fire? | 02:02:15 |
| raitobezarius | vs. garbage collecting and giving up on that data and preparing for the next set of data | 02:02:17 |
| raitobezarius | In reply to @nh2:matrix.org
For your colo, does it offer separate datacenters that isolated failure domains against fire? Nope | 02:02:27 |
| raitobezarius | If that was a goal, I would just send things in Glacier that I really care about, e.g. FODs | 02:02:56 |
| raitobezarius | In cache.nixos.org, there's cache :) | 02:03:02 |
| raitobezarius | Actually, cache.nixos.org was unavailable a lot of times | 02:03:20 |
