!zghijEASpYQWYFzriI:nixos.org

Hydra

343 Members
99 Servers

Load older messages


SenderMessageTime
2 Apr 2024
@lotte:chir.rs@lotte:chir.rs changed their profile picture.07:36:25
3 Apr 2024
@rick:matrix.ciphernetics.nl@rick:matrix.ciphernetics.nlManaged to fix a ca-derivations issue with hydra: https://github.com/NixOS/hydra/pull/137421:08:45
4 Apr 2024
@stablejoy:matrix.org@stablejoy:matrix.org joined the room.06:09:42
@fabaff:matrix.orgfabaff changed their display name from Fabian Affolter to fabaff.08:40:33
@darkwater4213:matrix.orgdarkwater4213

How feasible would it be to run Hydra as a distributed system? Say I had a few dozen or maybe even a hundred or more thin clients (ARM or RISC-V or whatever). Is it theoretically possible for Hydra to be distributed among those clients (assuming they're all on the same overarching network, of course) so none of them is doing much work or for long, but the whole thing is entirely bootstrapped (reminiscent of IPFS)?

And because this is distributed across an entire fleet, you can do some crazy Gentoo-level optimization... the prospect is very exciting indeed.

08:56:09
@darkwater4213:matrix.orgdarkwater4213Imagine: serverless fleet management. MDM... without a manager.08:56:34
@k900:0upti.meK900What do you even mean by "distributed system"?08:56:41
@k900:0upti.meK900Hydra is not a generic orchestrator like k8s08:56:49
@k900:0upti.meK900It builds stuff08:56:51
@k900:0upti.meK900If you don't need to build stuff, or need more than to build stuff, Hydra will not solve your problem08:57:03
@darkwater4213:matrix.orgdarkwater4213Distributed computation has been around since the 90's (at least). I'm asking how feasible it would be to distribute the computation necessary for Hydra. My theorycrafting brain says that the compilation would be the easy part, the orchestration and delivery would be more difficult. Delivery (that is, finding what unit in the fleet has a given resource and then pushing it out to all the units) seems like it'd be a good job for IPFS, but orchestration seems more difficult. Any ideas?08:59:39
@k900:0upti.meK900You are saying long words that don't mean things09:00:06
@k900:0upti.meK900What "computation necessary"?09:00:11
@k900:0upti.meK900 What do you actually want to do? 09:00:15
@darkwater4213:matrix.orgdarkwater4213
In reply to @k900:0upti.me
Hydra is not a generic orchestrator like k8s
I think you misunderstand. I'm not trying to treat Hydra as an orchestration tool. I'm asking how feasible it would be to orchestrate a large fleet of relatively low-resource clients (think along the lines of a generic tablet) to effectively function as a single Hydra instance or, barring that, multiple Hydra instances orchestrated by some distributed system I haven't come up with yet.
09:01:40
@k900:0upti.meK900What do you mean by "single Hydra instance"?09:01:55
@k900:0upti.meK900Do you want to use a bunch of shitty machines as builders?09:02:11
@k900:0upti.meK900That can be done easily09:02:18
@k900:0upti.meK900But you will need a central Hydra server09:02:23
@k900:0upti.meK900The actual build workers just need Nix09:02:50
@k900:0upti.meK900They don't run any Hydra09:02:54
@darkwater4213:matrix.orgdarkwater4213
In reply to @k900:0upti.me
But you will need a central Hydra server

No. A bunch of kinda okay machines.

The key here is no central server. Has that been done before? If not, how feasible is it?

09:02:58
@k900:0upti.meK900No09:03:02
@k900:0upti.meK900No central server will not happen09:03:05
@k900:0upti.meK900Without effectively rewriting Hydra entirely09:03:14
@k900:0upti.meK900And realistically it's just not a thing most people will want or need09:03:33
@darkwater4213:matrix.orgdarkwater4213
In reply to @k900:0upti.me
No central server will not happen
Why not? What does the central server provide that can't be decentralized?
09:03:42
@k900:0upti.meK900 It could be decentralized 09:03:57
@darkwater4213:matrix.orgdarkwater4213
In reply to @k900:0upti.me
And realistically it's just not a thing most people will want or need
It'd be a proof-of-concept, to be sure.
09:03:58
@k900:0upti.meK900It just won't be Hydra anymore09:04:02

Show newer messages


Back to Room ListRoom Version: 6