got it. is there anything I can help with? Sounds like there's an ongoing effort to refactor hostPlatform to be a "polycule". Is there anything I can do there?

I have a lot of machines with Nvidia GPUS that I use nix & nixOS with, so I would still like to help improve the system https://prayag.bhakar.org/000-00-0000/apollo-server.jpg

Does Asahi work on M2?

also yes! https://asahilinux.org/fedora/#device-support

If anyone has a decent modern GPU to test the flash-attention tests, please ping me.
The CUDA team's infra is not sufficent:

python3.13-flash-attention> FAILED tests/losses/test_cross_entropy.py::test_cross_entropy_loss[128256-0.9-0.7-True-0.01-True-False-dtype2] - torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1002.00 MiB. GPU 0 has a total capacity of 19.55 GiB of which 360.38 MiB is free. Including non-PyTorch memory, this process has 19.19 GiB memory in use. Of the allocated memory 18.10 GiB is allocated by PyTorch, and 925.39 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://docs.pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf)

Thanks in advance for your generosity

I'm pretty sure that would fit. Thanks a lot!

You'd need to add the following to your config:

      programs.nix-required-mounts = {
        enable = true;
        presets.nvidia-gpu.enable = true;
      };

Then

nix build github:GaetanLepage/nixpkgs/flash-attn#python3Packages.flash-attn.gpuCheck --cores 10

nix build --impure --cores 2 --expr '
    (import (builtins.getFlake "github:GaetanLepage/nixpkgs/flash-attn") {
      system = builtins.currentSystem;
      config = { allowUnfree = true; cudaSupport = true; };
    }).python3Packages.flash-attn.gpuCheck
  '

This should do it.

nix build --impure --expr '
    (import (builtins.getFlake "github:GaetanLepage/nixpkgs/flash-attn") {
      system = builtins.currentSystem;
      config = { allowUnfree = true; cudaSupport = true; };
    }).python3Packages.flash-attn.gpuCheck
  '

This should do it.

In reply to @hexa:lossy.network
in this economy?!

I mean... If only I had nix installed...

root@p4-r01-ct18:~# nvidia-smi
Wed May 27 16:10:23 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.21             Driver Version: 580.126.21     CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GB200                   On  |   00000008:01:00.0 Off |                    0 |
| N/A   45C    P0            170W / 1200W |       0MiB / 189471MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GB200                   On  |   00000009:01:00.0 Off |                    0 |
| N/A   45C    P0            153W / 1200W |       0MiB / 189471MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   2  NVIDIA GB200                   On  |   00000018:01:00.0 Off |                    0 |
| N/A   45C    P0            153W / 1200W |       0MiB / 189471MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   3  NVIDIA GB200                   On  |   00000019:01:00.0 Off |                    0 |
| N/A   45C    P0            176W / 1200W |       0MiB / 189471MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+