got it. is there anything I can help with? Sounds like there's an ongoing effort to refactor hostPlatform to be a "polycule". Is there anything I can do there?

I have a lot of machines with Nvidia GPUS that I use nix & nixOS with, so I would still like to help improve the system https://prayag.bhakar.org/000-00-0000/apollo-server.jpg

Does Asahi work on M2?

also yes! https://asahilinux.org/fedora/#device-support

If anyone has a decent modern GPU to test the flash-attention tests, please ping me.
The CUDA team's infra is not sufficent:

python3.13-flash-attention> FAILED tests/losses/test_cross_entropy.py::test_cross_entropy_loss[128256-0.9-0.7-True-0.01-True-False-dtype2] - torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1002.00 MiB. GPU 0 has a total capacity of 19.55 GiB of which 360.38 MiB is free. Including non-PyTorch memory, this process has 19.19 GiB memory in use. Of the allocated memory 18.10 GiB is allocated by PyTorch, and 925.39 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://docs.pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf)

Thanks in advance for your generosity

I'm pretty sure that would fit. Thanks a lot!

You'd need to add the following to your config:

      programs.nix-required-mounts = {
        enable = true;
        presets.nvidia-gpu.enable = true;
      };

Then

nix build github:GaetanLepage/nixpkgs/flash-attn#python3Packages.flash-attn.gpuCheck --cores 10

nix build --impure --cores 2 --expr '
    (import (builtins.getFlake "github:GaetanLepage/nixpkgs/flash-attn") {
      system = builtins.currentSystem;
      config = { allowUnfree = true; cudaSupport = true; };
    }).python3Packages.flash-attn.gpuCheck
  '

This should do it.

nix build --impure --expr '
    (import (builtins.getFlake "github:GaetanLepage/nixpkgs/flash-attn") {
      system = builtins.currentSystem;
      config = { allowUnfree = true; cudaSupport = true; };
    }).python3Packages.flash-attn.gpuCheck
  '

This should do it.

In reply to @hexa:lossy.network
in this economy?!