!CcTBuBritXGywOEGWJ:matrix.org

NixOS Binary Cache Self-Hosting

162 Members
About how to host a very large-scale binary cache and more55 Servers

Load older messages


SenderMessageTime
2 Mar 2024
@raitobezarius:matrix.orgraitobezariusLet's say, if HW manufacturers said: here's disks, here's servers (cheap in the context anyhow), and we could just pay colo, that'd be interesting02:06:30
@raitobezarius:matrix.orgraitobezariusetc.02:06:33
@nh2:matrix.orgnh2This is not what I mean. I mean that if you buy the hardware for a colo, it costs a fixed price. You could put it into your colo in France, or in Hetzner, where you would get proper HA.02:06:33
@raitobezarius:matrix.orgraitobezarius
In reply to @nh2:matrix.org
This is not what I mean. I mean that if you buy the hardware for a colo, it costs a fixed price. You could put it into your colo in France, or in Hetzner, where you would get proper HA.
This, I don't disagree
02:06:56
@raitobezarius:matrix.orgraitobezariusBut what's your point regarding that?02:07:07
@nh2:matrix.orgnh2My point is: Hetzner is currently the cheapest $/TB hardware rental on the planet. The only cheaper way (long term) is owning the hardware and putting it into colo. So I'm just pointing out, if you're considering colo, colo at Hetzner might be the way to go for the absolute cheapest and still reliable.02:08:52
@raitobezarius:matrix.orgraitobezariusI'm not extremely familiar with Hetzner colocation02:10:31
@raitobezarius:matrix.orgraitobezariusbut reading from that https://www.hetzner.com/colocation/02:10:35
@raitobezarius:matrix.orgraitobezarius assuming 1kW (constant) over one year, assuming 2 locations in Finland, at the announced electricity rates, this is something like 2×(119 + 0.2618€/kWh1kW1year) = 4827,87 € 02:12:28
@raitobezarius:matrix.orgraitobezarius * assuming 1kW (constant) over one year, assuming 2 locations in Finland, at the announced electricity rates, this is something like 2×(119 + 0.2618€/kWh*1kW*1year) = 4827,87 € 02:12:38
@raitobezarius:matrix.orgraitobezariusover one year02:12:45
@raitobezarius:matrix.orgraitobezariusfor 17U of 2×10A if I understand well02:12:55
@raitobezarius:matrix.orgraitobezariusand 1Gbps02:12:57
@raitobezarius:matrix.orgraitobezariusI guess an additional 92,82€/mo because of the traffic02:13:28
@raitobezarius:matrix.orgraitobezariusand then some classical disk replacement schema over N years taking into account remote hands costs which are probably a constant at this point02:14:05
@raitobezarius:matrix.orgraitobezariusso that's the sort of proposal I'd be interested reading if there was any Hetzner colocation idea02:14:25
@raitobezarius:matrix.orgraitobezariusthough this doesn't totally beat any random colocation because of the seemingly lack of custom peering options and the limited bandwidth02:14:56
@raitobezarius:matrix.orgraitobezariusunclear to me if hetzner can offer something better on that end02:15:09
@raitobezarius:matrix.orgraitobezariusoh shoot I forgot the 119€/mo02:15:33
@raitobezarius:matrix.orgraitobezariusbut yeah something like 8K€/year + ϵ anyway02:15:52
@raitobezarius:matrix.orgraitobezariusso still competitive compared to 1470€/mo because in 17U, you could fit like a ridiculous amount of disks via JBODs or whatever02:16:33
@raitobezarius:matrix.orgraitobezariusand have complete control over the tiering02:16:39
@raitobezarius:matrix.orgraitobezarius so assuming we do have the hardware, this is a compelling option and if we offer a proper complete proposal to the foundation and operations make sense and infrastructure agrees to it, it could be adopted I guess 02:18:24
@raitobezarius:matrix.orgraitobezarius * so assuming we do have the hardware, this is a compelling option and if we offer a proper complete proposal to the foundation and operations make sense and infrastructure folks agrees to it, it could be adopted I guess 02:18:30
@raitobezarius:matrix.orgraitobezariusand to give an example of hardware costs, I guess a netapp de6600 60x3.5" could cost something like 800EUR, you can fit 60×20TB disk in there, so 1.2PB raw capacity, 60 disks of 20TB will cost approximately 440€*60 ~ 26.4K EUR at list price, obviously, so something like 27K EUR, this can be spread over the 2 locations to avoid having 1.2PB × 2 needlessly (and the rest can be filled as the cache grows organically)02:24:08
@raitobezarius:matrix.orgraitobezariusI'm ignoring server costs because honestly you can find a R730 in a trash bin, put enough SAS cards and plug the JBOD in02:24:30
@raitobezarius:matrix.orgraitobezariustough question is whether flash is needed at all or not02:24:41
@raitobezarius:matrix.orgraitobezariusif so, this can add 5K-20K to the proposal02:24:55
@raitobezarius:matrix.orgraitobezariusactually hetzner seems to have proper connection options: https://docs.hetzner.com/robot/colocation/pricing02:32:47
@raitobezarius:matrix.orgraitobezariusthey were just hidden02:32:48

Show newer messages


Back to Room ListRoom Version: 10