Hello everyone!

I want to share some news regarding the progress yorik.sar and I made on the infra. Some were already communicated, but it doesn't work to summarize everything.

• We have been building and caching all cudaSupport-sensitive nixpkgs packages on both stable and unstable nixpkgs channels on our Hydra instance for a few weeks. You can check the cuda-packages-unstable and cuda-packages-stable jobsets.
• All gpuCheck instances across nixpkgs are now automatically and exhaustively collected and built in Hydra as well. See the cuda-gpu-checks-unstable and cuda-gpu-checks-stable jobsets. For a reminder, gpuChecks are derivations that run some package tests that need access to a physical GPU.
• Regarding package maintenance and updates, recent times were hectic as nixpkgs received several breaking changes since the beginning of 2026 (recursing into python314Packages, update to GCC 15 and more staging-next treats). Besides, here are notable ones:
  • [MERGED] onnxruntime update to 1.23.1
  • [MERGED] triton-bin 3.6.0 + torch-bin 2.10.0
  • [WIP] triton 3.6.0 + torch 2.10.0
  • [WIP] opencv 4.13.0. This one should allow us to migrate to CUDA 13 treewide.
  • [WIP] jax 0.9.0
  • [WIP] vllm 0.14.1
  • [MERGED] mistral-rs 0.7.0
• This is still an idea, but my plan is to create two cuda-specific nix channels: nixos-unstable-cuda and nixos-stable-cuda where we could ensure that a curated set of package builds and tests are succesful (release blockers). I would be glad to hearing your feedback on this idea.

We're trying our best to move things forward as fast as possible. Unfortunately, time and compute resources are limited, so it's never fast enough 😅

Have a nice day!

Hello everyone!

I want to share some news regarding the progress yorik.sar, connor (burnt/out) (UTC-8), SomeoneSerge (back on matrix) and I made recently. Some were already communicated, but it doesn't work to summarize everything.

• We have been building and caching all cudaSupport-sensitive nixpkgs packages on both stable and unstable nixpkgs channels on our Hydra instance for a few weeks. You can check the cuda-packages-unstable and cuda-packages-stable jobsets.
• All gpuCheck instances across nixpkgs are now automatically and exhaustively collected and built in Hydra as well. See the cuda-gpu-checks-unstable and cuda-gpu-checks-stable jobsets. For a reminder, gpuChecks are derivations that run some package tests that need access to a physical GPU.
• Regarding package maintenance and updates, recent times were hectic as nixpkgs received several breaking changes since the beginning of 2026 (recursing into python314Packages, update to GCC 15 and more staging-next treats). Besides, here are notable ones:
  • [MERGED] onnxruntime update to 1.23.1
  • [MERGED] triton-bin 3.6.0 + torch-bin 2.10.0
  • [WIP] triton 3.6.0 + torch 2.10.0
  • [WIP] opencv 4.13.0. This one should allow us to migrate to CUDA 13 treewide.
  • [WIP] jax 0.9.0
  • [WIP] vllm 0.14.1
  • [MERGED] mistral-rs 0.7.0
• This is still an idea, but my plan is to create two cuda-specific nix channels: nixos-unstable-cuda and nixos-stable-cuda where we could ensure that a curated set of package builds and tests are succesful (release blockers). I would be glad to hearing your feedback on this idea.

We're trying our best to move things forward as fast as possible. Unfortunately, time and compute resources are limited, so it's never fast enough 😅

Have a nice day!

Hello everyone!

I want to share some news regarding the progress yorik.sar, connor (burnt/out) (UTC-8), SomeoneSerge (back on matrix) and I made recently. Some were already communicated, but it doesn't work to summarize everything.

• We have been building and caching all cudaSupport-sensitive nixpkgs packages on both stable and unstable nixpkgs channels on our Hydra instance for a few weeks. You can check the cuda-packages-unstable and cuda-packages-stable jobsets.
• All gpuCheck instances across nixpkgs are now automatically and exhaustively collected and built in Hydra as well. See the cuda-gpu-checks-unstable and cuda-gpu-checks-stable jobsets. For a reminder, gpuChecks are derivations that run some package tests that need access to a physical GPU.
• Regarding package maintenance and updates, recent times were hectic as nixpkgs received several breaking changes since the beginning of 2026 (recursing into python314Packages, update to GCC 15 and more staging-next treats). Besides, here are notable ones:
  • [MERGED] onnxruntime update to 1.23.1
  • [MERGED] triton-bin 3.6.0 + torch-bin 2.10.0
  • [WIP] triton 3.6.0 + torch 2.10.0
  • [WIP] opencv 4.13.0. This one should allow us to migrate to CUDA 13 treewide.
  • [WIP] jax 0.9.0
  • [WIP] vllm 0.14.1
  • [MERGED] mistral-rs 0.7.0
• This is still an idea, but my plan is to create two cuda-specific nix channels: nixos-unstable-cuda and nixos-25.11-cuda where we could ensure that a curated set of package builds and tests are succesful (release blockers). I would be glad to hearing your feedback on this idea.

We're trying our best to move things forward as fast as possible. Unfortunately, time and compute resources are limited, so it's never fast enough 😅

Have a nice day!

RE: effort to migrate towards cuda 13 treewide

magma fails to build with cuda 13. I opened https://github.com/NixOS/nixpkgs/pull/487064 to fix it.

Yes it's currently on JetPack 5.1.3. I'm using 11cb3517b3af6af300dd6c055aeda73c9bf52c48 from nixpkgs (still 25.05 ;)). As for opencv:

          opencv = pkgs.opencv.override {
            enableCudnn = true;
            cudaPackages = pkgs.cudaPackages_11_4;
          };

and I'm using this for the nixpkgs config:

    {
      config = {
        cudaSupport = true;
        allowUnfree = true;
        allowBroken = true;
      };
      overlays = [
        (import rust-overlay)
      ];
    }

ok, i did make some progress, turns out libcuda has a couple of more dependencies on the jetson

ldd /run/opengl-driver/lib/libcuda.so linux-vdso.so.1 (0x0000ffffa1fb3000) libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffffa0796000) libnvrm_gpu.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_gpu.so (0x0000ffffa0729000) libnvrm_mem.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_mem.so (0x0000ffffa0711000) libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffffa0666000) libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffffa0652000) librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000ffffa063a000) libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffffa0609000) libnvrm_sync.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_sync.so (0x0000ffffa05f2000) libnvrm_host1x.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_host1x.so (0x0000ffffa05d1000) /lib/ld-linux-aarch64.so.1 (0x0000ffffa1f83000) libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000ffffa05b1000) libnvsocsys.so => /usr/lib/aarch64-linux-gnu/tegra/libnvsocsys.so (0x0000ffffa059d000) libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000ffffa03b8000) libnvsciipc.so => /usr/lib/aarch64-linux-gnu/tegra/libnvsciipc.so (0x0000ffffa0393000) libnvrm_chip.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_chip.so (0x0000ffffa037f000) libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000ffffa035b000)

I set LD_LIBRARY_PATH such that the linker is able to load those which did work for a small sample c program that I wrote. Need to try this workaround for the "big app" now ;)

ok, i did make some progress, turns out libcuda has a couple of more dependencies on the jetson

``
dd /run/opengl-driver/lib/libcuda.so linux-vdso.so.1 (0x0000ffffa1fb3000) libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffffa0796000) libnvrm_gpu.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_gpu.so (0x0000ffffa0729000) libnvrm_mem.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_mem.so (0x0000ffffa0711000) libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffffa0666000) libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffffa0652000) librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000ffffa063a000) libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffffa0609000) libnvrm_sync.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_sync.so (0x0000ffffa05f2000) libnvrm_host1x.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_host1x.so (0x0000ffffa05d1000) /lib/ld-linux-aarch64.so.1 (0x0000ffffa1f83000) libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000ffffa05b1000) libnvsocsys.so => /usr/lib/aarch64-linux-gnu/tegra/libnvsocsys.so (0x0000ffffa059d000) libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000ffffa03b8000) libnvsciipc.so => /usr/lib/aarch64-linux-gnu/tegra/libnvsciipc.so (0x0000ffffa0393000) libnvrm_chip.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_chip.so (0x0000ffffa037f000) libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000ffffa035b000)

I set LD\_LIBRARY\_PATH such that the linker is able to load those which did work for a small sample c program that I wrote. Need to try this workaround for the "big app" now ;)

ok, i did make some progress, turns out libcuda has a couple of more dependencies on the jetson

dd /run/opengl-driver/lib/libcuda.so linux-vdso.so.1 (0x0000ffffa1fb3000) libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffffa0796000) libnvrm\_gpu.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm\_gpu.so (0x0000ffffa0729000) libnvrm\_mem.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm\_mem.so (0x0000ffffa0711000) libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffffa0666000) libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffffa0652000) librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000ffffa063a000) libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffffa0609000) libnvrm\_sync.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm\_sync.so (0x0000ffffa05f2000) libnvrm\_host1x.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm\_host1x.so (0x0000ffffa05d1000) /lib/ld-linux-aarch64.so.1 (0x0000ffffa1f83000) libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000ffffa05b1000) libnvsocsys.so => /usr/lib/aarch64-linux-gnu/tegra/libnvsocsys.so (0x0000ffffa059d000) libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000ffffa03b8000) libnvsciipc.so => /usr/lib/aarch64-linux-gnu/tegra/libnvsciipc.so (0x0000ffffa0393000) libnvrm\_chip.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm\_chip.so (0x0000ffffa037f000) libgcc\_s.so.1 => /lib/aarch64-linux-gnu/libgcc\_s.so.1 (0x0000ffffa035b000)

I set LD_LIBRARY_PATH such that the linker is able to load those which did work for a small sample c program that I wrote. Need to try this workaround for the "big app" now ;)