| 14 May 2024 |
 raitobezarius | Maybe my dumb view on this is I need to get an Ampere Altra Q80-30 somewhere, wire this up to gigabit (or even 100mbps) and then we get faster CI to some extent | 00:56:36 |
| raitobezarius | Then there's things like balancing out aarch64 capacity between Darwin and Linux | 00:56:54 |
| raitobezarius | Then there's things like improving scheduling as a whole | 00:57:16 |
| raitobezarius | Etc etc | 00:57:17 |
 samrose | In reply to @raitobezarius:matrix.org
There's also some critical performance improvements on the scheduling side of things here you are referring to the scheduling of ci builds is that right? | 00:57:39 |
| raitobezarius | In reply to @samrose:matrix.org
here you are referring to the scheduling of ci builds is that right? More generally how a Nix aware CI component will decide to place a build on a piece of compute (policy, binpacking, etc.) | 00:58:37 |
| raitobezarius | It can be Nix scheduler, can be Hydra scheduler, there's multiple possible designs and paths | 00:59:03 |
| samrose | hmm, so like you envision that this ci component might become part of nix or hydra I see | 00:59:12 |
| samrose | cool! | 00:59:15 |
 Qyriad | hey raito | 00:59:21 |
| Qyriad | weren't you going to bed =P | 00:59:29 |
| raitobezarius | I am going to get yelled at | 00:59:37 |
| samrose | Keepin you up! | 00:59:43 |
| raitobezarius | In reply to @qyriad:katesiria.org
weren't you going to bed =P I am in bed tho | 00:59:51 |
| Qyriad | …okay fair | 00:59:58 |
| *  puck stare at the clock | 01:00:00 |
| raitobezarius | In reply to @samrose:matrix.org
Keepin you up! No problem :-) | 01:00:03 |
| * puck * stare at the clock (why is it 3am) | 01:00:04 |
| raitobezarius | In reply to @puck:puck.moe
stare at the clock (why is it 3am) You too ig | 01:00:16 |
| raitobezarius | OK now 30 mn of Isekai manga | 01:00:40 |
| raitobezarius | And I will sleep myself away | 01:00:45 |
| samrose | In reply to @raitobezarius:matrix.org
Maybe my dumb view on this is I need to get an Ampere Altra Q80-30 somewhere, wire this up to gigabit (or even 100mbps) and then we get faster CI to some extent fwiw in some of these kooky nix startup companies that i worked with over the years, we started out using self-hosted ci builders (especially for darwin, but also for arm in one case) and it actually worked fine, although of course you gotta be careful about who can trigger it.
I do have one friend over at equinix metal and I don't know if they would help out one day in the future but it's possible.
I am excited about a branch of nix where people are starting to try and fix the issues, document things and evolve ecosystem and core.
| 01:06:01 |
 jkachmar | In reply to @raitobezarius:matrix.org
More generally how a Nix aware CI component will decide to place a build on a piece of compute (policy, binpacking, etc.) we sorta have this at $DAYJOB, my coworker has slowly been making progress on extracting it for open sourcing | 01:30:22 |
| jkachmar | he ended up just ignoring the remote build protocol tho; our system is basically a centralized, work-stealing queue of machines that progress thru a build graph of derivations w/ additional metadata for hinting at machine sizing & weighting for scheduling priority | 01:32:27 |
| jkachmar | * he ended up just ignoring the remote build protocol tho; our system is basically a centralized, work-stealing queue fashioned after the graph of whatever derivations get submitted.
since it's all coordinated w/ Postgres, builders progress thru the graph by querying for available derivations based on builder features, weights, etc. | 01:34:18 |
| jkachmar | more intelligent binpacking would be nice, but i think actually just running a bunch of garbage & applying weights based on historical characteristics works kinda great esp. if you have global deduplication of builds | 01:35:01 |
| jkachmar | but to tie it back to something lix-relevant: man... a lot of work basically went into hitting problems w/ nix-the-application and just having to work around more stuff | 01:41:30 |
| samrose | In reply to @jkachmar:matrix.org
he ended up just ignoring the remote build protocol tho; our system is basically a centralized, work-stealing queue fashioned after the graph of whatever derivations get submitted.
since it's all coordinated w/ Postgres, builders progress thru the graph by querying for available derivations based on builder features, weights, etc. yeah I also often wondered about just short circuiting by storing derivation data in postgres. Maybe that is more possible with content addressed derivations? I work on a product that is hosted postgres with a postgest api and was experimenting with wrapping nix with cli that can talk to postgrest and store data like this. It also seems like maybe you wouldn't need binpacking if you had a simple agent on the build machine that can just reveal the resources available | 01:49:53 |
 DarkKronicle | (Let me know if I should ask this question somewhere else). Is there a way to debug why Lix is building from source and not using the binary cache? It installs just fine, just wondering if it's on my end. I added the substituter to my substituter list and double checked the public key. (My current version is 2.90.0-beta.1-lixpre20240506-b6799ab). Each time it builds it gives me a different hash, so maybe I have a weird dependency issue? | 02:11:04 |
 leo60228 | In reply to @raitobezarius:matrix.org
Maybe my dumb view on this is I need to get an Ampere Altra Q80-30 somewhere, wire this up to gigabit (or even 100mbps) and then we get faster CI to some extent apparently asrock rack recently launched a "normal" ampere altra board | 02:11:14 |
