!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

274 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda55 Servers

Load older messages

SenderMessageTime
25 Oct 2025
@daniel-fahey:matrix.orgDaniel FaheyThanks! I think it's ready to merge then, I've marked the PR ready to review12:43:44
@glepage:matrix.orgGaétan Lepage I was away from my computer, but I managed to run nixpkgs-review successfully. Good job Daniel Fahey!
I've seen that happysalade merged the PR. 		18:17:00
@daniel-fahey:matrix.orgDaniel Fahey No problem, thanks for running nixpkgs-review and good to know TorchRL and KServe are OK 18:24:42
@daniel-fahey:matrix.orgDaniel Faheybtw I'm having a quick look into https://hydra.nixos-cuda.org/build/1784 18:29:42
@glepage:matrix.orgGaétan LepageOh good!18:48:06
@angleangleside:matrix.orgasa is there a reason packages like tensorrt require a manual installation and then running nix-store --add-fixed sha256 TensorRT-10.9.0.34.Linux.x86_64-gnu.cuda-12.8.tar.gz to put them in the nix store? 21:41:07
@angleangleside:matrix.orgasaas opposed to directly downloading the tarball (eg. from https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.9.0/tars/TensorRT-10.9.0.34.Linux.x86_64-gnu.cuda-12.8.tar.gz) under the relevant license21:41:45
@daniel-fahey:matrix.orgDaniel FaheyIDK I'm just guessing licensing?21:42:17
@angleangleside:matrix.orgasayeah it seems like a weird "nvidia requires you to sign in and agree to their eula" thing21:42:41
@angleangleside:matrix.orgasabut i don't think(?) it violates their license if private groups that already have agreed to the eula override the tensorrt package to fetch from the url anyway21:43:18
@daniel-fahey:matrix.orgDaniel FaheyThey relaxed a lot of their law fare bullshit recently21:43:38
@angleangleside:matrix.orgasaa related question - it appears the tensorrt in nixpkgs is labelled only for x86_64 (possibly because nvidia only offers generic linux binaries for x64), but they offer a tarball for ubuntu 24, would i just be able to override the source for the arm build and let patchelf handle the rest?21:44:53
@angleangleside:matrix.orgasa * a related question - it appears the tensorrt in nixpkgs is labelled only for x86_64 (possibly because nvidia only offers generic linux binaries for x64), but they only offer a tarball for arm targeting ubuntu 24, would i just be able to override the source for the arm build and let patchelf handle the rest? 21:45:20
@angleangleside:matrix.orgasa * a related question - it appears the tensorrt in nixpkgs is labelled only for x86_64 (possibly because nvidia only offers generic linux binaries for x64), but they only offer a tarball for arm targeting ubuntu 24. would i just be able to override the source for the arm build and let patchelf handle the rest? 21:45:38
@daniel-fahey:matrix.orgDaniel FaheyTry it out, looks like it needs a new maintainer anyway. I nominate you https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/tensorrt/default.nix21:46:58
@angleangleside:matrix.orgasasounds good, perhaps i'll try to figure out how to use nixpkgs' cuda infra first21:49:07
@angleangleside:matrix.orgasaalso for anyone with possibly more knowledge on this, tensorrt downloads are public endpoints (https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.9.0/tars/TensorRT-10.9.0.34.Linux.x86_64-gnu.cuda-12.8.tar.gz) that you can get without authenticating would it be a license violation to directly include that in nixpkgs with the relevant package license instead of via separately joining their developer program?22:24:21
@daniel-fahey:matrix.orgDaniel FaheyCan't remember where I saw this link, I though it was in here, anyone know of any other discussion / context re: Nix? It reports Flox are signed up? https://developer.nvidia.com/blog/developers-can-now-get-cuda-directly-from-their-favorite-third-party-platforms/22:59:54
@daniel-fahey:matrix.orgDaniel FaheyDuckDuckGo was better at searching the Discouse, heh, https://discourse.nixos.org/t/nix-flox-nvidia-opening-up-cuda-redistribution-on-nix/6918923:02:41
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) asa: only the 10-series are available without a login.
They also don't publish the URLs anywhere public except on their TensorRT OSS repo (only for current releases). You can see the URLs for previous releases assuming you log in to NVIDIA's website.
See https://github.com/NixOS/nixpkgs/pull/437723; I've got a helper bash script which can be used to generate a manifest file similar to what NVIDIA provides for their other binary archives. 		23:03:12
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)As for having only x86_64-linux releases -- that's mostly been because releases for Jetson were few and far in between and I don't have access to hopper or blackwell to verify ARM releases function correctly23:04:10
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)As I'm working on https://github.com/nixos-cuda/cuda-legacy/pull/1 I'll be making manifests for older versions of TensorRT available publicly23:06:27
@angleangleside:matrix.orgasaoh i see do the arm builds only run on jetsons?23:06:38
@daniel-fahey:matrix.orgDaniel Fahey* https://developer.nvidia.com/blog/developers-can-now-get-cuda-directly-from-their-favorite-third-party-platforms/23:06:55
@daniel-fahey:matrix.orgDaniel Fahey* https://discourse.nixos.org/t/nix-flox-nvidia-opening-up-cuda-redistribution-on-nix/6918923:07:01
@angleangleside:matrix.orgasaalso this is very useful thanks, i didn't know something like cuda-legacy existed23:08:50
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) NVIDIA has historically published two "arm" builds: linux-sbsa and linux-aarch64. These both map to Nixpkgs' aarch64-linux platform, but linux-sbsa is for server-grade ARM and linux-aarch64 for Jetsons.
Going forward, it seems new Jetson devices will also use linux-sbsa (e.g., the Jetson Thor). It remains to be seen whether older Jetson devices will be supported on CUDA 13. 		23:09:20
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)It's still very much a work in progress and best-effort, so beware 👻23:10:04
@daniel-fahey:matrix.orgDaniel Fahey

After diffing the inputDrvs

diff <(nix derivation show --impure -I nixpkgs=https://github.com/daniel-fahey/nixpkgs/archive/3d2db9755e7815937fb7b8f089fad9b44bc416d8.tar.gz --expr 'with import <nixpkgs> { }; python313Packages.vllm' | jq -r '.[] | .inputDrvs | keys[]' | sort -t'-' -k2) <(nix derivation show --impure -I nixpkgs=https://github.com/daniel-fahey/nixpkgs/archive/6a08e6bb4e46ff7fcbb53d409b253f6bad8a28ce.tar.gz --expr 'with import <nixpkgs> { }; python313Packages.vllm' | jq -r '.[] | .inputDrvs | keys[]' | sort -t'-' -k2)
37c37
< zv7b6x3i1k6vgmlx4ll74a676znpp19l-python3.13-outlines-1.2.3.drv
---
> b0yp229czgvgsxay9vw8mq0xraa0w94l-python3.13-outlines-1.2.3.drv
23:17:03
@daniel-fahey:matrix.orgDaniel Fahey *

After diffing the inputDrvs

$ diff <(nix derivation show --impure -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/3d2db9755e7815937fb7b8f089fad9b44bc416d8.tar.gz --expr 'with import <nixpkgs> { }; python313Packages.vllm' | jq -r '.[] | .inputDrvs | keys[]' | sort -t'-' -k2) <(nix derivation show --impure -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/6a08e6bb4e46ff7fcbb53d409b253f6bad8a28ce.tar.gz --expr 'with import <nixpkgs> { }; python313Packages.vllm' | jq -r '.[] | .inputDrvs | keys[]' | sort -t'-' -k2)
37c37
< zv7b6x3i1k6vgmlx4ll74a676znpp19l-python3.13-outlines-1.2.3.drv
---
> b0yp229czgvgsxay9vw8mq0xraa0w94l-python3.13-outlines-1.2.3.drv

$ diff <(nix derivation show --impure -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/3d2db9755e7815937fb7b8f089fad9b44bc416d8.tar.gz --expr 'with import <nixpkgs> { }; python313Packages.outlines' | jq -r '.[] | .inputDrvs | keys[]' | sort -t'-' -k2) <(nix derivation show --impure -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/6a08e6bb4e46ff7fcbb53d409b253f6bad8a28ce.tar.gz --expr 'with import <nixpkgs> { }; python313Packages.outlines' | jq -r '.[] | .inputDrvs | keys[]' | sort -t'-' -k2)
15c15
< fcsb1iizwb50wiag876xzj9kpd8vpa9s-python3.13-jax-0.8.0.drv
---
> 2vxlyjm486swfp89iwsacyx52rslq4xj-python3.13-jax-0.8.0.drv

$ diff <(nix derivation show --impure -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/3d2db9755e7815937fb7b8f089fad9b44bc416d8.tar.gz --expr 'with import <nixpkgs> { }; python313Packages.jax' | jq -r '.[] | .inputDrvs | keys[]' | sort -t'-' -k2) <(nix derivation show --impure -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/6a08e6bb4e46ff7fcbb53d409b253f6bad8a28ce.tar.gz --expr 'with import <nixpkgs> { }; python313Packages.jax' | jq -r '.[] | .inputDrvs | keys[]' | sort -t'-' -k2)
0a1
> 521737jxrb1akanvm7df9zmx3236rwc6-af5c211d49f3b99447db2252d2cc2b8e0fb54d1c.patch?full_index=1.drv

That last patch is the latest fix to JAX. Maybe a red herring, but I'm running a fresh build of vLLM to see if the new JAX version (and hence Outlines) breaks vLLM.

23:21:15

Show newer messages

Back to Room ListRoom Version: 9