!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

286 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda58 Servers

Load older messages

SenderMessageTime
16 Feb 2025
@aidalgol:matrix.orgaidalgolI know I added TensorRT to nixpkgs, but I've fallen behind with some of the derivation refactoring.18:56:59
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) Yeah it just needs a cleanup
Honestly need to move to the 10.x series which don’t have a login preventing download — I’ve got those packaged out of tree
Speaking of, in the out of tree stuff I’ve been working on I’m postponing the CUDA 12.8 update and instead going to be making a PR to merge it back into Nixpkgs 		19:43:16
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) That will remove support for CUDA 11 (and PowerPC, but that’s more of an NVIDIA thing), but Jetson devices running JetPack 5-based OS (like JetPack NixOS) will still be supported by way of cuda_compat up to 12.2 19:46:47
17 Feb 2025
@zimbatm:numtide.comJonas ChevalierWe have https://opencollective.com/europe that acts as a fiscal host. In practice it means that I pay the invoices, and then get reimbursed by the legal entity. Since it's all subscribtion we haven't encountered any ownership issue so far, so I'm not sure how it would work if we were to buy hardware. If that helps, I don't mind creating a non-profit with a mission that's aligned with our goals. It might take a month or so.20:21:58
18 Feb 2025
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)

SomeoneSerge (UTC+U[-12,12]): I'm gonna need guidance on how best to approach getting the work I've done in cuda-packages upstreamed. I re-organized the tree so it looks more like Nixpkgs (https://github.com/connorbaker/cuda-packages) and I have a branch where I copied stuff over into Nixpkgs (https://github.com/NixOS/nixpkgs/compare/master...ConnorBaker:nixpkgs:feat/cudaPackages-modules-round-two) but it's a 20k line diff.
In terms of the order of upstreaming stuff, I was thinking:

  • out of package-set things, like utility packages/setup hooks and tests for those
  • setup hooks for the package set (this includes wrapping NVCC such that we get rid of backendStdenv
  • introduction of cudaLib which is full of utility functions and types necessary for everything else
  • introduction of rewritten modules, top-level/cuda-packages.nix, and the remainder of the changes
    , I could start with the foundational out-of-package-set stuff I made which I use for setup hooks.
01:04:04
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) *

SomeoneSerge (UTC+U[-12,12]): I'm gonna need guidance on how best to approach getting the work I've done in cuda-packages upstreamed. I re-organized the tree so it looks more like Nixpkgs (https://github.com/connorbaker/cuda-packages) and I have a branch where I copied stuff over into Nixpkgs (https://github.com/NixOS/nixpkgs/compare/master...ConnorBaker:nixpkgs:feat/cudaPackages-modules-round-two) but it's a 20k line diff.
In terms of the order of upstreaming stuff, I was thinking:

  • out of package-set things, like utility packages/setup hooks and tests for those
  • setup hooks for the package set (this includes wrapping NVCC such that we get rid of backendStdenv
  • introduction of cudaLib which is full of utility functions and types necessary for everything else
  • introduction of rewritten modules, top-level/cuda-packages.nix, and the remainder of the changes
01:04:13
@indoor_squirrel:matrix.orgindoor_squirrelRedacted or Malformed Event01:08:08
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) *

SomeoneSerge (UTC+U[-12,12]): I'm gonna need guidance on how best to approach getting the work I've done in cuda-packages upstreamed. I re-organized the tree so it looks more like Nixpkgs (https://github.com/connorbaker/cuda-packages) and I have a branch where I copied stuff over into Nixpkgs (https://github.com/NixOS/nixpkgs/compare/master...ConnorBaker:nixpkgs:feat/cudaPackages-modules-round-two) but it's a 20k line diff.
In terms of the order of upstreaming stuff, I was thinking:

  1. out of package-set things, like utility packages/setup hooks and tests for those
  2. setup hooks for the package set (this includes wrapping NVCC such that we get rid of backendStdenv
  3. introduction of cudaLib which is full of utility functions and types necessary for everything else
  4. introduction of rewritten modules, top-level/cuda-packages.nix, and the remainder of the changes
01:08:26
@stites:matrix.org@stites:matrix.org left the room.12:56:25
@justbrowsing:matrix.orgKevin Mittman (EOY sleep)Anyone here planning to attend GPU Technology Conference (GTC) next month in Bay Area?21:37:48
19 Feb 2025
@ss:someonex.netSomeoneSerge (back on matrix) I see, makes sense. Yeah we can start looking into establishing another non-profit (another, as in in addition to the Foundation). I think we can afford ourselves a bit of reflection time to decide on the structure and jurisdiction 09:34:55
@zimbatm:numtide.comJonas ChevalierOr we could keep track of who owns which hardware and keep it simple09:45:43
@zimbatm:numtide.comJonas ChevalierSomething to consider09:46:01
@ss:someonex.netSomeoneSerge (back on matrix)

Let's start with some high-level questions before delving into the code?

setup hooks for the package set (this includes wrapping NVCC such that we get rid of backendStdenv)

Removing backendStdenv for wrappers implies we commit t hey o support mixed stdenvs. This might not even be supported by upstream projects (llvm, gcc), and you've already discovered the incompatibility for LTO. I'm thinking if there are any downsides to introducing the wrapper code and still using the wrapper with the single (no mixing) stdenv in Nixpkgs. That way out-of-tree derivations already can (ab)use the wrapper without knowing it exists and without knowing they actually need to replace the stdenv

introduction of rewritten modules, top-level/cuda-packages.nix, and the remainder of the changes

What's the gist of the rewrite?

09:52:44
@ss:someonex.netSomeoneSerge (back on matrix) Something like, an individual buys a piece of equipment and gets a reimbursement from the opencollective fund? Totally fine by me, the question is, is this transparent enough for users and sponsors. In addition to off-boarding issues, we should sooner or later figure out transparency of physical access to the infra (for all you know I'm a Russian sleeper agent) 10:01:21
@ss:someonex.netSomeoneSerge (back on matrix) * Something like, an individual buys a piece of equipment and gets a reimbursement from the opencollective fund? Totally fine by me (EDIT: except we can't write VAT off?), the question is, is this transparent enough for users and sponsors. In addition to off-boarding issues, we should sooner or later figure out transparency of physical access to the infra (for all you know I'm a Russian sleeper agent) 10:02:23
@zimbatm:numtide.comJonas ChevalierWe can maintain a page like OpenWRT does, with who owns what machine: https://openwrt.org/infrastructure#servers10:55:50
20 Feb 2025
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)I’ll get on a gist of a rewrite, as well as design tradeoffs in offering a wrapped cuda_nvcc, cudaStdenv, or changing the global stdenv when cudaSupport is enabled (to provide both LTO and to prevent any possibilities of linking against multiple copies of glibc/glibcxx)06:31:21
21 Feb 2025
@tillerino:matrix.org@tillerino:matrix.org joined the room.17:14:13
23 Feb 2025
@angleangleside:matrix.orgasa joined the room.09:07:57
25 Feb 2025
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)
In reply to @connorbaker:matrix.org
I’ll get on a gist of a rewrite, as well as design tradeoffs in offering a wrapped cuda_nvcc, cudaStdenv, or changing the global stdenv when cudaSupport is enabled (to provide both LTO and to prevent any possibilities of linking against multiple copies of glibc/glibcxx)
I am still doing this, among other things 		00:09:17
26 Feb 2025
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Anyone free to review https://github.com/NixOS/nixpkgs/pull/383214 and https://github.com/NixOS/nixpkgs/pull/383511?15:34:10
@ss:someonex.netSomeoneSerge (back on matrix)In my backlog but might take me until the end of the week. I wish I could switch earlier15:54:52
@hexa:lossy.networkhexawondering if anyone here cares about onnxruntime a bit17:52:35
@hexa:lossy.networkhexaenabling openvino support breaks the tests for some reason, since it cannot find libonnxruntime_providers_shared.so in $out/lib/17:52:55
@hexa:lossy.networkhexa * enabling openvino support breaks the tests for some reason, since it cannot find libonnxruntime_providers_shared.so in $out/lib/ 17:53:06
@hexa:lossy.networkhexa * enabling openvino support breaks the tests for some reason, since it cannot find libonnxruntime_providers_shared.so in $out/lib/ 17:53:11
@hexa:lossy.networkhexaso I added a condition to disable the tests with openvinoSupport17:53:39
@hexa:lossy.networkhexa 
>>> rt.get_available_providers()
['OpenVINOExecutionProvider', 'CPUExecutionProvider']
17:53:42
@hexa:lossy.networkhexa* so I added a condition to disable the tests with openvinoSupport, which gets me to here:17:53:54

Show newer messages

Back to Room ListRoom Version: 9