| 14 Apr 2023 |
 aciceri | but it works with root@rock5b.fleet now | 08:13:58 |
 @linus:schreibt.jetzt | I don't know off the top of my head. I have a layer of indirection:
buildMachinesFiles = [(pkgs.writeText "hydra-builders" ''
localhost aarch64-linux,armv7l-linux - 16 16
localhost-big-parallel aarch64-linux,armv7l-linux - 1 1 big-parallel,kvm,nixos-test
oak-hydra x86_64-linux,i686-linux - 16 16
oak-big-parallel x86_64-linux,i686-linux - 1 1 big-parallel,kvm,nixos-test
'')];
and
programs.ssh.extraConfig = ''
Host localhost-big-parallel
Hostname localhost
Host oak-hydra oak-big-parallel
Hostname oak.sphalerite.tech
User hydra
'';
| 08:14:41 |
| @linus:schreibt.jetzt | ah, there's a nix.buildMachines.*.sshUser option | 08:15:22 |
| @linus:schreibt.jetzt | so probably set that to root :) | 08:15:27 |
 das_j | we have something long the lines of ssh://nix-builder@helsinki-hydra-builder05.wg aarch64-linux /run/secrets/hydra/builders.key 8 8 kvm,nixos-test,big-parallel,benchmark - - | 08:16:09 |
| aciceri | without setting sshUser what is used? consider that I was already able to remote build using the CLI | 08:17:43 |
| @linus:schreibt.jetzt | no user is specified and ssh will use the local username | 08:17:59 |
| @linus:schreibt.jetzt | and the nix daemon runs as root so it coincidentally works | 08:18:12 |
| aciceri | btw do you use localhost as "remote" builder? | 08:19:17 |
| @linus:schreibt.jetzt | yeah, I do that so I can run 16 normal builds separately from the 1 big-parallel build | 08:19:45 |
| @linus:schreibt.jetzt | hydra special-cases "localhost" and skips the SSH connection though | 08:20:19 |
| aciceri | for this reason you have both localhost and localhost-big-parallel, right? | 08:21:03 |
| aciceri | yes I remembered that there were some hard coded conditions about being localhost in the hydra source | 08:21:32 |
| @linus:schreibt.jetzt | yes | 08:21:34 |
| aciceri | it caused me a several headaches | 08:21:38 |
| aciceri | * yes I remember that there were some hard coded conditions about being localhost in the hydra source | 08:21:49 |
| @linus:schreibt.jetzt | 🙃 | 08:21:53 |
| @linus:schreibt.jetzt | as a workaround you can use foo.localhost :p | 08:22:04 |
| aciceri | or 127.0.0.1 or was it hardcoded too? | 08:23:00 |
| aciceri | btw setting also sshUser to root still doesn't work | 08:23:11 |
| @linus:schreibt.jetzt | yeah but that forces IPv4 and that makes me sad :D | 08:23:11 |
| @linus:schreibt.jetzt | huh | 08:23:29 |
| aciceri | this is my /etc/machines:
ssh://root@rock5b.fleet aarch64-linux /etc/ssh/ssh_host_ed25519_key 6 2 nixos-test,benchmark,big-parallel,kvm - -
| 08:23:52 |
| @linus:schreibt.jetzt | oh, maybe it's trying to use the SSH host key as specified, and of course hydra-queue-runner can't read it | 08:24:25 |
| @linus:schreibt.jetzt | maybe the logs of hydra-queue-runner have some helpful infomration | 08:24:38 |
| aciceri | Apr 14 10:24:56 mothership hydra-queue-runner[3329939]: checking the queue for builds > 371...
Apr 14 10:24:57 mothership hydra-queue-runner[3329939]: got notification: builds restarted
Apr 14 10:24:57 mothership hydra-queue-runner[3329939]: checking the queue for builds > 0...
Apr 14 10:24:57 mothership hydra-queue-runner[3329939]: loading build 371 (trotten:master:aarch64-linux.treefmt)
Apr 14 10:24:57 mothership hydra-queue-runner[3329939]: aborting unsupported build step '/nix/store/cf0kfmf12cqrcvjjsp48zbzp20nw1q5q-treefmt.toml.drv' (type 'aarch64-linux')
Apr 14 10:24:57 mothership hydra-queue-runner[3329939]: marking build 371 as failed
Apr 14 10:25:07 mothership hydra-queue-runner[3329939]: checking the queue for builds > 371...
| 08:25:20 |
| aciceri | i can try to change permissions for a minute, just to see it this is the problem | 08:25:50 |
| @linus:schreibt.jetzt | that will break sshing into your machine fwiw | 08:26:04 |
| @linus:schreibt.jetzt | I think | 08:26:09 |
| aciceri | why? | 08:26:25 |
