no worries SomeoneSerge (matrix works sometimes) I'm grateful for any time folks are taking out of their day to help me out. I figured the conversation may be more fruitful in the matrix channel

Binfmt/qemu is not "cross-compilation"

ah, sorry I was under the impression it was close enough. Since option A is off the table, do options B and C have any legs? I understand that there is a process to secure more compute, but can that compute come from volunteers like me? From my understanding even with just a Jetson nano, it can start compiling other aarch64 packages until more capable compute is secured

Now to what is a priority to us,

oh I see, so have the GPU definitions be part of the core hostPlatform instead of treated like added on accelerators. Does this mean there are plans to also add core hostPlatform support for other accelerators like TPUs and FPGAs? Is there someplace I can read up on this body of work and try to figure out how I can help? I'm happy to scrap my PR if there is a more established north start being pursued by the team

While a Jetson can indeed build packages for aarch64-linux, having a single one would not enable anything.
The amount of (decently fast) ARM cores needed to start enabling aarch64-linux jobsets on hydra.nixos-cuda.org is ~64 at the very extreme minimum.

We appreciate the proposition, but until we get access to something resembling a serious build capacity for this platform, we won't spend time supporting it.

got it. is there anything I can help with? Sounds like there's an ongoing effort to refactor hostPlatform to be a "polycule". Is there anything I can do there?

I have a lot of machines with Nvidia GPUS that I use nix & nixOS with, so I would still like to help improve the system https://prayag.bhakar.org/000-00-0000/apollo-server.jpg

Does Asahi work on M2?

also yes! https://asahilinux.org/fedora/#device-support

If anyone has a decent modern GPU to test the flash-attention tests, please ping me.
The CUDA team's infra is not sufficent:

python3.13-flash-attention> FAILED tests/losses/test_cross_entropy.py::test_cross_entropy_loss[128256-0.9-0.7-True-0.01-True-False-dtype2] - torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1002.00 MiB. GPU 0 has a total capacity of 19.55 GiB of which 360.38 MiB is free. Including non-PyTorch memory, this process has 19.19 GiB memory in use. Of the allocated memory 18.10 GiB is allocated by PyTorch, and 925.39 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://docs.pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf)

Thanks in advance for your generosity

I'm pretty sure that would fit. Thanks a lot!

You'd need to add the following to your config:

      programs.nix-required-mounts = {
        enable = true;
        presets.nvidia-gpu.enable = true;
      };

Then

nix build github:GaetanLepage/nixpkgs/flash-attn#python3Packages.flash-attn.gpuCheck --cores 10