| 14 May 2024 |
|  joegilkes joined the room. | 10:03:50 |
 samrose | In reply to @raitobezarius:matrix.org
It can be Nix scheduler, can be Hydra scheduler, there's multiple possible designs and paths I am exploring creating a replacement for nomad, vault, and consul with a combo of postgrest, and an elixir agent on machine + elixir middleware doing dynamic scheduling (targeting only microvm.nix ). I am actually prototyping it with supabase, but trying to make it so that it’ll just work with postgrest too. I am making temporal.io an optional component in this too (to handle complicated queue and workflow). Might not be useful for hydra ng but sharing in any case | 12:14:39 |
 K900 | I don't think it's the kind of thing we need for Hydra | 12:17:26 |
| K900 | But sounds interesting | 12:17:31 |
| K900 | But Hydra is not a typical cluster workload because it's a huge DAG of batch jobs | 12:18:06 |
| K900 | That have wildly different resource profiles | 12:18:36 |
| K900 | And ideally you want to optimize for data locality on top | 12:18:46 |
|  vivian joined the room. | 12:19:15 |
| samrose | In reply to @k900:0upti.me
But Hydra is not a typical cluster workload because it's a huge DAG of batch jobs Yes it seems like a hydra-like system might end up needing to be more closely integrated with nix evaluation etc also there may be many things that have been discussed about evolving nix itself that I’m not up to speed on, that could lead to a better hydra later | 12:56:33 |
| samrose | (Or some kind of build orchestration). | 12:57:13 |
| K900 | Right now Hydra evaluates everything ahead of time and then just distributes the builds and it works reasonably well | 12:57:18 |
| K900 | It won't work with dyndrvs but dyndrvs also don't work anyway so | 12:57:31 |
| samrose | In reply to @k900:0upti.me
It won't work with dyndrvs but dyndrvs also don't work anyway so I wasn’t even aware of dyndrvs | 13:00:51 |
| K900 | That's because they don't work so nothing uses them | 13:01:09 |
| K900 | Has anyone actually tried building dyndrvs on Hydra? | 13:05:29 |
| K900 | I'm like 60% sure it will explode | 13:05:41 |
| K900 | But I'm curious | 13:05:45 |
| samrose | In reply to @k900:0upti.me
Right now Hydra evaluates everything ahead of time and then just distributes the builds and it works reasonably well Yeah (not talking in this response about hydra for lix, but just in general) I got interested in the idea of running nix-eval-jobs and then storing the data emitted from that in postgrest, and then eventually storing the corresponding successful build and cache location data in postgrest, and then having a tool that wraps nix be able to check that before doing an install and using the cache data if trusted, etc (short circuiting some of the expensive nix computations if possible). Probably I am not the first with this idea I imagine. | 13:09:54 |
| K900 | That is basically Hydra | 13:10:16 |
| samrose | Heh | 13:10:50 |
| samrose | In reply to @k900:0upti.me
That is basically Hydra It seemed simpler to iterate on for me to use postgrest, which automatically creates an api as you add data, takes care of authentication etc. also for me this made it nicer to search across package sets via pg full text search. And then the db could hold an archive of all packages built, the source, etc and it might make it simpler to build applications etc off of that for companies or people who would use it as a package source (and they might also use a cli or ui wrapper or create their own etc) | 13:15:55 |
| K900 | I don't like Postgrest personally but I'm not going to attempt to convince you | 13:16:39 |
| samrose | Yeah and this is just discussion for me, not trying to convince any of you to adopt my thing 😀 | 13:18:38 |
| samrose | But I work at supabase, and happen to absolutely love postgrest | 13:19:35 |
| samrose | In reply to @k900:0upti.me
I don't like Postgrest personally but I'm not going to attempt to convince you Maybe not here but I am probably going to be working on postgrest this year, and would value your perspective on what you don’t like about it, to understand what problems people might see with it | 13:22:12 |
| samrose | (But might be better to pm me if you feel like sharing). | 13:22:46 |
| K900 | I just don't think it's a good approach for most things, because you usually end up needing more than just Postgres eventually and then you start having to add stuff on top and meh | 13:24:41 |
| K900 | Like, if you're sure you can run everything on top of postgrest, sure | 13:25:03 |
|  Mia :) joined the room. | 13:25:19 |
| K900 | But every time I've seen it used it ended up eventually not being enough and then people started building tools around it that just made things more complicated | 13:25:50 |
