Thanks Prayag Bhakar for your suggestions.
At the time, our CI capacity only allows us to build ~10% of our "target jobsets" for x86_64 alone.
This means that cross-building for aarch64 is definitely not feasible with the current hardware.

We are actively working in the background to secure "sponsorships" and get a legitimate compute capacity, but this is not done yet.
Supporting Jetson architectures is definitely on the list of things we would like to do, but we are way too much hardware-bound for it.

Hey, sorry for disappearing on the github ticket, v limited bandwidth currently.

The first and most important point, elaborating on Gaétan's reply: our infra currently lacks aarch64 builders, and frankly we as of now haven't even a fraction of the x8664 capacity that we need. We are currently working on securing proper funding for the hardware and for the general effort, and first and foremost for reducing the harm and the extra load that CUDA imposes on the rest of Nixpkgs maintainers. It's been in the works for almost two years now. Recently our entire team, including our new "manager" member @dhofer:matrix.org, has been fully dedicating to making this happen, but, while there's been some very modest progress, it's going to take a while before we start testing and caching Jetsons or in fact anything besides the vanilla x86-64-linux nixpkgs simply due to cashflow considerations. We are actively looking for companies willing to properly pay for the service of testing and keeping the Jetson ecosystem "green" & functional here upstream in Nixpkgs, but so far there's no ETAs for this specific effort. In our personal projects we normally use very different kind of hardware, so it's not a priority for any of the maintainers.

Regarding your other messages, just some clarifications:

1. Binfmt/qemu is not "cross-compilation". Nixpkgs does have cross capabilities, albeit less stable than native builds, and we are also very interest in making cross compilation possible for CUDA projects (it currently isn't, mostly because nvidia is making it artificially hard), but it's not a priority for any of us, and we haven't found any interested customers or sponsors yet either.
2. Binfmt builds in CI are indeed not impossible, and Connor and I had burned quite a bit of energy by trying them out in the old Hercules-based CI... It's been an incredible pain, besides also harming reproducibility and all. Jetsons without paying consumers and without our own research projects needing them is not something we'd want to spend our currently extremely limited and scarce compute on, I'm afraid.
3. Re: the PR, as I mentioned we moved from nix-community infra to our own, and we had to temporarily pause consuming the release-cuda.nix file (which, though, remains the authoritative source, and which is also what's being built by Flox). I'd honestly rather first do the hard work of removing backendStdenv and config.cuda*, because then we wouldn't need to modify release-lib in this first place. That said, if this were a priority, we could hack jetsons in in the release-cuda file, e.g. to get them built by Flox (I forget if they do aarch64 though)

Now to what is a priority to us, fot example: I'm happy to brainstorm with anyone about how do we get rid of the backendStdenv thing and how to cleanly model coprocessors/accelerators in the elaborated-system structure, whether to make coprocessors part of a system "quadruple" (contrast to triple), and whether the specific cudaCapabilities and/or rocm gpuTargets should become a part of... well, I suppose, the system "polycule".
I know @lt1379:matrix.org and @tomberek:matrix.org , besides the CUDA Team have been pondering about the same questions.

no worries SomeoneSerge (matrix works sometimes) I'm grateful for any time folks are taking out of their day to help me out. I figured the conversation may be more fruitful in the matrix channel

Binfmt/qemu is not "cross-compilation"

ah, sorry I was under the impression it was close enough. Since option A is off the table, do options B and C have any legs? I understand that there is a process to secure more compute, but can that compute come from volunteers like me? From my understanding even with just a Jetson nano, it can start compiling other aarch64 packages until more capable compute is secured

Now to what is a priority to us,

oh I see, so have the GPU definitions be part of the core hostPlatform instead of treated like added on accelerators. Does this mean there are plans to also add core hostPlatform support for other accelerators like TPUs and FPGAs? Is there someplace I can read up on this body of work and try to figure out how I can help? I'm happy to scrap my PR if there is a more established north start being pursued by the team

While a Jetson can indeed build packages for aarch64-linux, having a single one would not enable anything.
The amount of (decently fast) ARM cores needed to start enabling aarch64-linux jobsets on hydra.nixos-cuda.org is ~64 at the very extreme minimum.

We appreciate the proposition, but until we get access to something resembling a serious build capacity for this platform, we won't spend time supporting it.

got it. is there anything I can help with? Sounds like there's an ongoing effort to refactor hostPlatform to be a "polycule". Is there anything I can do there?

I have a lot of machines with Nvidia GPUS that I use nix & nixOS with, so I would still like to help improve the system https://prayag.bhakar.org/000-00-0000/apollo-server.jpg

Does Asahi work on M2?

also yes! https://asahilinux.org/fedora/#device-support

If anyone has a decent modern GPU to test the flash-attention tests, please ping me.
The CUDA team's infra is not sufficent:

python3.13-flash-attention> FAILED tests/losses/test_cross_entropy.py::test_cross_entropy_loss[128256-0.9-0.7-True-0.01-True-False-dtype2] - torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1002.00 MiB. GPU 0 has a total capacity of 19.55 GiB of which 360.38 MiB is free. Including non-PyTorch memory, this process has 19.19 GiB memory in use. Of the allocated memory 18.10 GiB is allocated by PyTorch, and 925.39 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://docs.pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf)

Thanks in advance for your generosity

I'm pretty sure that would fit. Thanks a lot!

You'd need to add the following to your config:

      programs.nix-required-mounts = {
        enable = true;
        presets.nvidia-gpu.enable = true;
      };

Then

nix build github:GaetanLepage/nixpkgs/flash-attn#python3Packages.flash-attn.gpuCheck --cores 10