!CcTBuBritXGywOEGWJ:matrix.org

NixOS Binary Cache Self-Hosting

158 Members
About how to host a very large-scale binary cache and more54 Servers

Load older messages


SenderMessageTime
24 Oct 2024
@quentinmit:matrix.orgquentin
In reply to @joerg:thalheim.io
quentin: you seem to have configured a build hook in your nix.conf: https://nix.dev/manual/nix/2.24/command-ref/conf-file.html#conf-build-hook

I haven't (intentionally, at least). This is my complete nix.conf:

      extra-experimental-features = nix-command flakes
      substituters = ${s3Url} https://cache.nixos.org/
      require-sigs = false
05:40:04
@joerg:thalheim.ioMic92derivation-goal.cc05:40:58
@joerg:thalheim.ioMic92https://github.com/NixOS/nix/blob/3db75b00602a7a937c1e983e01c071100102c18c/src/libstore/build/derivation-goal.cc#L115705:41:09
@joerg:thalheim.ioMic92Ok. I think this not like post-build-hook05:41:21
@joerg:thalheim.ioMic92This is maybe used when doing remote builds?05:41:33
@joerg:thalheim.ioMic92https://github.com/NixOS/nix/blob/3db75b00602a7a937c1e983e01c071100102c18c/src/libstore/unix/build/hook-instance.cc#L4805:42:28
@joerg:thalheim.ioMic92 quentin: I would add more -vvvvv to your build to get more low-level output 05:42:59
@joerg:thalheim.ioMic92potentially also needs to be done in nix-daemon as this one might actually start this binary05:43:19
@quentinmit:matrix.orgquentin I'm not sure how to thread -vvvv through nix-fast-build 05:43:32
@joerg:thalheim.ioMic92use nix build directly on one of the builds that fails05:43:52
@quentinmit:matrix.orgquentinUnfortunately that succeeds05:43:59
@joerg:thalheim.ioMic92mhm. ok. maybe some sort of resource depletion?05:44:38
@joerg:thalheim.ioMic92 Does it happen if you do nix-fast-build -j 1? 05:45:13
@joerg:thalheim.ioMic92that will run less stuff in parallel05:45:21
@joerg:thalheim.ioMic92 Otherwise strace -f is your friend for these kind of errors. You might want to log it to a file 05:45:49
@joerg:thalheim.ioMic92 if you can install sysdig via programs.sysdig.enable = true in nixos, than try sysdig -c stderr 05:46:17
@joerg:thalheim.ioMic92for sysdig you will need root05:47:02
@quentinmit:matrix.orgquentinAs I said above the builds are running on Google Cloud Build05:48:16
@quentinmit:matrix.orgquentinso no NixOS, no nix-daemon05:48:22
@quentinmit:matrix.orgquentin Just invoking nix-fast-build from within a nix-built Docker container 05:48:46
@joerg:thalheim.ioMic92Ok. Don't have much experience with google cloud build. Good luck05:50:16
@quentinmit:matrix.orgquentinAlso there's no remote builders involved05:50:27
@quentinmit:matrix.orgquentin I was wondering if there was a race condition between the nix-eval-jobs process and the nix-build processes in local store manipulations 05:51:06
@quentinmit:matrix.orgquentin
In reply to @joerg:thalheim.io
Does it happen if you do nix-fast-build -j 1?
-j 1 doesn't help; it still reliably fails
05:51:24
@quentinmit:matrix.orgquentin Is there any way to thread -v or --debug through nix-fast-build? I couldn't find anything equivalent that I could put in nix.conf, for instance. 05:51:59
@joerg:thalheim.ioMic92not at the moment, you would have to clone the repo, patch it in and do nix run05:53:34
@quentinmit:matrix.orgquentinAlright. That sounds like a tomorrow problem. :)05:55:20
@quentinmit:matrix.orgquentinThanks for your thoughts05:55:22
25 Oct 2024
@lholh:matrix.org@lholh:matrix.org joined the room.03:54:50
@lholh:matrix.org@lholh:matrix.org left the room.22:42:39

Show newer messages


Back to Room ListRoom Version: 10