!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

332 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda63 Servers

Load older messages

SenderMessageTime
27 May 2026
@hexa:lossy.networkhexain this economy?!16:04:38
@glepage:matrix.orgGaétan Lepage 
nix build --impure --cores 2 --expr '
    (import (builtins.getFlake "github:GaetanLepage/nixpkgs/flash-attn") {
      system = builtins.currentSystem;
      config = { allowUnfree = true; cudaSupport = true; };
    }).python3Packages.flash-attn.gpuCheck
  '

This should do it.

16:05:28
@glepage:matrix.orgGaétan Lepage * 
nix build --impure --expr '
    (import (builtins.getFlake "github:GaetanLepage/nixpkgs/flash-attn") {
      system = builtins.currentSystem;
      config = { allowUnfree = true; cudaSupport = true; };
    }).python3Packages.flash-attn.gpuCheck
  '

This should do it.

16:05:59
@berrij:fairydust.spaceBerriJ
In reply to @hexa:lossy.network
in this economy?!
It's not my private one unfortunately 😅
But I'm the admin and currently there is no workload on that thing. 		16:07:57
@glepage:matrix.orgGaétan Lepage

I mean... If only I had nix installed...

root@p4-r01-ct18:~# nvidia-smi
Wed May 27 16:10:23 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.21             Driver Version: 580.126.21     CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GB200                   On  |   00000008:01:00.0 Off |                    0 |
| N/A   45C    P0            170W / 1200W |       0MiB / 189471MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GB200                   On  |   00000009:01:00.0 Off |                    0 |
| N/A   45C    P0            153W / 1200W |       0MiB / 189471MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   2  NVIDIA GB200                   On  |   00000018:01:00.0 Off |                    0 |
| N/A   45C    P0            153W / 1200W |       0MiB / 189471MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   3  NVIDIA GB200                   On  |   00000019:01:00.0 Off |                    0 |
| N/A   45C    P0            176W / 1200W |       0MiB / 189471MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
16:10:33
@hexa:lossy.networkhexamakes you wonder who we are building cuda support for16:12:00
@glepage:matrix.orgGaétan LepageNot for the owners of those GPUs unfortunately 🥲16:20:08
@berrij:fairydust.spaceBerriJIn my case I'm working at a German University and the server is used by a team of around 9 researchers :) 16:33:56
@hexa:lossy.networkhexapretty sure Gaetan works at some French university 😆16:37:26
@glepage:matrix.orgGaétan LepageNot anymore. (French universities don't have such fancy GPUs) 🫠16:38:38
@berrij:fairydust.spaceBerriJThe build is running now :)17:16:42
@ss:someonex.netSomeoneSerge (matrix works sometimes)Can't you nix in container?18:26:52
@ss:someonex.netSomeoneSerge (matrix works sometimes)Not TUM?18:27:57
@ss:someonex.netSomeoneSerge (matrix works sometimes) Not the OS group? I'd be hyped yo learn that somebody in academia/hpc/rse community actually uses nixpkgs cuda, because so far I've been getting the vibes that only the enterprise cares, and all these eurohpc/CSC/yada yada are completely unapproachable and dead set on their easybuild lmod workflows... 18:33:29
@berrij:fairydust.spaceBerriJUniversity of Duisburg-Essen, not TUM But it's really not that big of a deal. The economics faculty has its own little IT department, they bought some servers for machine learning of which our Chair was able to get one and we asked them to install nixos on that for us cause we use nixos since 2 years on all of our machines. That's essentially the full story, there is not that much support for NixOS besides me pushing it and my Boss seeing the advantages and sometimes proudly talking about our infra 😅18:55:53
@ss:someonex.netSomeoneSerge (matrix works sometimes)Shooting in the dark but anything that could be done or reprioritized on our side to potentially help the lab's story?19:48:31
@berrij:fairydust.spaceBerriJWell the biggest point is the cache. Currently we obtain pytorch and other ml packages from pypi cause it has the CUDA binaries packaged directly. I we can't really risk getting cache misses and triggering a 5 hour recompilation on my colleagues machines. And setting up our own binary cache is also not trivial, we are working from home a lot and the machines are only connected to the university vpn on demand. I've read that there is this flox cache now, but I also read that this does not strictly follow nixos-unstable.20:35:31
@berrij:fairydust.spaceBerriJBy the way the build is still running it's at the `pytestCheckPhase` of flash attention and causes a good 60gb of VRAM usage at the moment. I'll call it a day and report on the status tomorrow morning 🙂20:37:16
@busti:leitstelle511.net@busti:leitstelle511.net left the room.21:16:57
28 May 2026
@glepage:matrix.orgGaétan Lepage CUDA 13.3 is out: https://developer.nvidia.com/blog/nvidia-cuda-13-3-enhances-gpu-development-with-tile-programming-in-c-compiler-autotuning-and-python-updates/ 07:16:08
@glepage:matrix.orgGaétan Lepage *

CUDA 13.3 is out: https://developer.nvidia.com/blog/nvidia-cuda-13-3-enhances-gpu-development-with-tile-programming-in-c-compiler-autotuning-and-python-updates/

PR: https://github.com/NixOS/nixpkgs/pull/525130

07:26:31
@berrij:fairydust.spaceBerriJ

Its still running but it shows an error:

[1/10/11 built] building python3.13-flash-attention-2.8.3 (buildPhase): [4/73] /nix/store/50fi3x00m5ksrpcmc4pbkh57h9dhl8ls-cuda12.9-cuda_nvcc-12.9.86/bin/nvcc -MD -MF /build/source/build/temp.linux-x86_64-cpython-313/csrc/flash_attn/src/flash_fwd_hdim64_bf16_causal_sm80.o.d -I/build/source/csrc/flash_attn -I/build/source/csrc/flash_attn/src -I/build/source/csrc/cutlass/include -I/nix/store/291svvk6bn3mfw0k57lp8d2plhdb1k46-python3.13-torch-2.11.0/lib/pyth[1/10/1[1/10/11 built] building python3.13-flash-attention-2.8.3 (buildPhase): [14/73] /nix/store/50fi3x00m5ksrpcmc4pbkh57h9dhl8ls-cuda12.9-cuda_nvcc-12.9.86/bin/nvcc -MD -MF /build/source/build/temp.linux-x86_64-cpython-313/csrc/flash_attn/src/flash_fwd_hdim32_bf16_sm80.o.d -I/build/source/csrc/flash_attn -I/build/source/csrc/flash_attn/src -I/build/source/csrc/cutlass/include -I/nix/store/291svvk6bn3mfw0k57lp8d2plhdb1k46-python3.13-torch-2.11.0/lib/pyt[1/10/11 built[1/10/11 built] building python3.13-flash-attention-2.8.3 (pytestCheckPhase): /nix/store/291svvk6bn3mfw0k57lp8d2plhdb1k46-python3.13-torch-2.11.0/lib/python3.13/site-packages/torch/cuda/random.py:126: AcceleratorErr[1/10/11 built] building python3.13-flash-attention-2.8.3 (pytestCheckPhase): /nix/store/291svvk6bn3mfw0k57lp8d2plhdb1k46-python3.13-torch-2.11.0/lib/python3.13/site-packages/torch/cuda/random.py:126: AcceleratorError
07:43:05
@berrij:fairydust.spaceBerriJ *

Its still running but it shows an error:

[1/10/11 built] building python3.13-flash-attention-2.8.3 (pytestCheckPhase): /nix/store/291svvk6bn3mfw0k57lp8d2plhdb1k46-python3.13-torch-2.11.0/lib/python3.13/site-packages/torch/cuda/random.py:126: AcceleratorError
07:43:34
@glepage:matrix.orgGaétan Lepage

Thanks for the feedback. The CUDA team is trying hard to make the community cache approved by Nvidia.
Also, we're working on ramping up our infra to build, test and cache more derivations, faster and for more platforms.
We have some plans to provide "cuda" nixpkgs channels that would guarantee certain packages to be cached.

We'll announce things as soon as they happen, but be sure that we take those requests very seriously

09:30:44
@prince213:matrix.orgprince213I wonder if people actually do LLM inference with CUDA packages from Nixpkgs10:32:45
@grw00:matrix.orggrw00tried to run this and earlyoom killed my desktop :D10:35:02
@prince213:matrix.orgprince213But anyway I’m working on packaging SGLang and trying to learn CUDA in Nixpkgs (the hard way I suppose)10:35:29
@prince213:matrix.orgprince213https://github.com/NixOS/nixpkgs/pull/52514110:35:33
@glepage:matrix.orgGaétan LepageI warned you ;)11:44:54
@glepage:matrix.orgGaétan LepageHaha nice! Feel free to ping me for review or if you need some help11:45:58

Show newer messages

Back to Room ListRoom Version: 9