!kFJOpVCFYFzxqjpJxm:nixos.org

Nix HPC

80 Members
Nix for High Perfomance Computing clusters19 Servers

You have reached the beginning of time (for this room).


SenderMessageTime
27 Mar 2023
@ryantalo:matrix.orgRyan Swart joined the room.09:41:42
29 Mar 2023
@ss:someonex.netSomeoneSerge (back on matrix) joined the room.19:39:45
@ss:someonex.netSomeoneSerge (back on matrix) Hi! Just wanted to mention that torchWithRocm is going to require allowUnfree = true when https://github.com/NixOS/nixpkgs/pull/222273 gets merged. I think this needs to be addressed upstream in openai/triton (they unconditionally download and vendor a copy of ptxas from cuda), but I haven't the capacity to handle this now 19:54:53
31 Mar 2023
@ryantalo:matrix.orgRyan SwartKind of a meta question - I am trying to run static nix on an HPC cluster and am running into some roadblocks (with a custom path and disabling the sandbox as much as possible I am still running into permission denied errors - can't chroot or run bubblewrap); given that I have the ability to run arbitrary binaries - is there a way to disable all "special" path modification etc in such a way that common HPC access limitations can allow nix to still build derivations?02:28:46
@ryantalo:matrix.orgRyan SwartI have looked into both static-nix and nix-portable, but neither seem to be a drop-in solution for my case02:29:36
2 Apr 2023
@jboy:utwente.ioJohn ✒️ryantalo: have you tried something like this? https://www.jboy.space/blog/nix-on-hpc.html11:56:18
1 Apr 2023
@ss:someonex.netSomeoneSerge (back on matrix) Anyone tried running running Nix from Singularity and binding /nix/store? The cluster at my uni won't enable user namespaces, so no unchroot and no bubblewrap for me 23:44:29
@ss:someonex.netSomeoneSerge (back on matrix) singularity run --bind /abc:/efg something.sif works, but I want to use singularity-tools.buildImage and that puts e.g. the shell into /nix/store, which then is hidden by the --bind =' 23:45:35
@ss:someonex.netSomeoneSerge (back on matrix) And singularity run --overlay is, again, disabled on the cluster xD 23:45:55
@ss:someonex.netSomeoneSerge (back on matrix)
In reply to @ryantalo:matrix.org
Kind of a meta question - I am trying to run static nix on an HPC cluster and am running into some roadblocks (with a custom path and disabling the sandbox as much as possible I am still running into permission denied errors - can't chroot or run bubblewrap); given that I have the ability to run arbitrary binaries - is there a way to disable all "special" path modification etc in such a way that common HPC access limitations can allow nix to still build derivations?

disable all "special" path modification etc

Hi, which path modifications are you referring to?

23:46:27
2 Apr 2023
@ryantalo:matrix.orgRyan SwartThat describes using chroot - which I don't have permissions to do; nix seems to get to a point of reading/writing source files, and takes forever to work through them (like 5 mins+, log example here: https://gist.github.com/ryanswrt/3ebf02bd10e25319f153014698a14c63) and then eventually dies13:40:48
3 Apr 2023
@ryantalo:matrix.orgRyan Swart It ends up dying with error: setting up a private mount namespace: Operation not permitted 00:50:04
@jb:vk3.wtf@jb:vk3.wtf
In reply to @ss:someonex.net
singularity run --bind /abc:/efg something.sif works, but I want to use singularity-tools.buildImage and that puts e.g. the shell into /nix/store, which then is hidden by the --bind ='
i use singularity on a hpc to bind in a nix store, but not in conjunction with a singularity image as since i have a nix store in the container i just use the nix in that store
01:19:37
@ss:someonex.netSomeoneSerge (back on matrix)
In reply to @jb:vk3.wtf
i use singularity on a hpc to bind in a nix store, but not in conjunction with a singularity image as since i have a nix store in the container i just use the nix in that store
Is it a prepoppulated store, or a writable image?
11:15:52

Show newer messages


Back to Room ListRoom Version: 9