11 Oct 2024 |
Tristan Ross | Oh | 04:57:04 |
emily | * "waiting for outputs", is what I always hear. | 04:57:06 |
Mindavi | Isn't that compression on the builders? | 05:11:06 |
Tristan Ross | In reply to @rick:matrix.ciphernetics.nl Isn't that compression on the builders? No | 05:11:24 |
Tristan Ross | https://github.com/NixOS/hydra/issues/553 | 05:11:35 |
Mindavi | Ah, like that | 05:14:43 |
Mindavi | Ok now it makes sense again :) | 05:14:55 |
vcunat | I'm fairly certainly that the compression of results on the central machine is what's limiting the throughput almost all the time, as the farm is currently. | 06:11:34 |
emily | an EPYC with better thermals seems like it'd be a good investment, then? | 06:12:06 |
vcunat | We have some bigger aarch64 in the farm already, so alternatively we could try that "for free". | 06:13:08 |
emily | wouldn't that run into the atomics and platform purity problems wrt the evaluator? | 06:13:59 |
vcunat | Right. It might be risky in some ways. | 06:15:46 |
emily | I take it setting up a separate Hydra instance to test it would be onerous – we'd need to just switch over? | 06:17:32 |
vcunat | I don't see a little down-time as an issue (for hydra.nixos.org) | 06:20:38 |
emily | I more meant it sounds like a pain to revert if it goes wrong | 06:22:16 |
emily | (but maybe it's not) | 06:22:32 |
vcunat | No need to wipe the old machine too soon. And it seems unlikely that we'd want to revert completely. | 06:23:14 |
tomberek | In reply to @emilazy:matrix.org if the Foundation will pay for a month then it seems worth it to just do it? Not too hard. We're spending similar amounts on meetups/pizza. Infra team should have a discretionary budget (no?), otherwise throw a submission into this repo similar to this: https://github.com/NixOS/foundation/issues | 06:39:11 |
emily | I didn't get the impression that the infra team has a budget at all | 06:39:33 |
emily | though I don't know how the finances work at all :) | 06:39:44 |
emily | In reply to @vcunat:matrix.org No need to wipe the old machine too soon. And it seems unlikely that we'd want to revert completely. I think we should do whatever you think is best for Hydra out of trying an existing AArch64 machine and getting an EPYC to try. | 06:40:24 |
emily | (without consideration of the costs, since they seem trivial compared to the potential benefit) | 06:40:42 |
vcunat | Keeping it at x86 seems better in our current situation (less risks/complications). | 06:53:55 |
vcunat | We might consider splitting it up. At least the queue-runner away from the public web (hydra.nixos.org). I don't feel too comfortable having the signing keys on same machine as a public web. I think these only communicate through the DB which is on a separate machine already. | 06:55:56 |
fricklerhandwerk | In reply to @joerg:thalheim.io @fricklerhandwerk:matrix.org: where is this security tracker deployed just now? I think it would help if Erethon would have a chat with us, so we can make sure it get integrated with the rest of the infra to some extent. http://sectracker.nixpkgs.lahfa.xyz/
Config here: https://github.com/Nix-Security-WG/nix-security-tracker/tree/main/staging
| 07:28:08 |
Gaétan Lepage | Redacted or Malformed Event | 07:30:30 |
fricklerhandwerk | In reply to @emilazy:matrix.org I didn't get the impression that the infra team has a budget at all The infra team can get expenses reimbursed out of the foundation's regular budget. The problem is that in the past there was no process to do anything else. tomberek and I have been pushing to have discretionary budgets for teams, but there needs to be someone on the other end to make those decisions. | 07:30:34 |
emily | gotcha | 07:33:51 |
emily | In reply to @vcunat:matrix.org We might consider splitting it up. At least the queue-runner away from the public web (hydra.nixos.org). I don't feel too comfortable having the signing keys on same machine as a public web. I think these only communicate through the DB which is on a separate machine already. maybe the website could continue being on the x86 server? | 07:34:24 |
emily | or, right, the idea was to upgrade the x86 | 07:34:39 |