!kFJOpVCFYFzxqjpJxm:nixos.org

Nix HPC

78 Members
Nix for High Perfomance Computing clusters19 Servers

You have reached the beginning of time (for this room).


SenderMessageTime
28 Feb 2023
@bootstrapper:matrix.org@bootstrapper:matrix.org joined the room.05:43:18
14 Mar 2023
@pederbs:pvv.ntnu.nopbsds joined the room.13:15:12
24 Mar 2023
@htran:matrix.orghtran joined the room.02:20:37
25 Mar 2023
@valconius:matrix.org@valconius:matrix.org joined the room.23:44:55
@rbutani:matrix.orgrbutani joined the room.23:55:19
27 Mar 2023
@ryantalo:matrix.orgRyan Swart joined the room.09:41:42
29 Mar 2023
@ss:someonex.netSomeoneSerge (back on matrix) joined the room.19:39:45
@ss:someonex.netSomeoneSerge (back on matrix) Hi! Just wanted to mention that torchWithRocm is going to require allowUnfree = true when https://github.com/NixOS/nixpkgs/pull/222273 gets merged. I think this needs to be addressed upstream in openai/triton (they unconditionally download and vendor a copy of ptxas from cuda), but I haven't the capacity to handle this now 19:54:53
31 Mar 2023
@ryantalo:matrix.orgRyan SwartKind of a meta question - I am trying to run static nix on an HPC cluster and am running into some roadblocks (with a custom path and disabling the sandbox as much as possible I am still running into permission denied errors - can't chroot or run bubblewrap); given that I have the ability to run arbitrary binaries - is there a way to disable all "special" path modification etc in such a way that common HPC access limitations can allow nix to still build derivations?02:28:46
@ryantalo:matrix.orgRyan SwartI have looked into both static-nix and nix-portable, but neither seem to be a drop-in solution for my case02:29:36
2 Apr 2023
@jboy:utwente.ioJohn ✒️ryantalo: have you tried something like this? https://www.jboy.space/blog/nix-on-hpc.html11:56:18
1 Apr 2023
@ss:someonex.netSomeoneSerge (back on matrix) Anyone tried running running Nix from Singularity and binding /nix/store? The cluster at my uni won't enable user namespaces, so no unchroot and no bubblewrap for me 23:44:29
@ss:someonex.netSomeoneSerge (back on matrix) singularity run --bind /abc:/efg something.sif works, but I want to use singularity-tools.buildImage and that puts e.g. the shell into /nix/store, which then is hidden by the --bind =' 23:45:35
@ss:someonex.netSomeoneSerge (back on matrix) And singularity run --overlay is, again, disabled on the cluster xD 23:45:55
@ss:someonex.netSomeoneSerge (back on matrix)
In reply to @ryantalo:matrix.org
Kind of a meta question - I am trying to run static nix on an HPC cluster and am running into some roadblocks (with a custom path and disabling the sandbox as much as possible I am still running into permission denied errors - can't chroot or run bubblewrap); given that I have the ability to run arbitrary binaries - is there a way to disable all "special" path modification etc in such a way that common HPC access limitations can allow nix to still build derivations?

disable all "special" path modification etc

Hi, which path modifications are you referring to?

23:46:27
2 Apr 2023
@ryantalo:matrix.orgRyan SwartThat describes using chroot - which I don't have permissions to do; nix seems to get to a point of reading/writing source files, and takes forever to work through them (like 5 mins+, log example here: https://gist.github.com/ryanswrt/3ebf02bd10e25319f153014698a14c63) and then eventually dies13:40:48
3 Apr 2023
@ryantalo:matrix.orgRyan Swart It ends up dying with error: setting up a private mount namespace: Operation not permitted 00:50:04
@jb:vk3.wtf@jb:vk3.wtf
In reply to @ss:someonex.net
singularity run --bind /abc:/efg something.sif works, but I want to use singularity-tools.buildImage and that puts e.g. the shell into /nix/store, which then is hidden by the --bind ='
i use singularity on a hpc to bind in a nix store, but not in conjunction with a singularity image as since i have a nix store in the container i just use the nix in that store
01:19:37

Show newer messages


Back to Room ListRoom Version: 9