!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

303 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda60 Servers

Load older messages


SenderMessageTime
9 Mar 2026
@kaya:catnip.eekaya 𖤐

Im currently in process of upstreaming the nixos module for tabbapi https://github.com/NixOS/nixpkgs/pull/498281
Does anyone know how i would go about setting the default package? In my nixos config i use the module like this right now, i always override the package:

    services.tabbyapi = {
      enable = true;
      package = pkgs.pkgsCuda.tabbyapi;
    };

I feel like it might be bad to have the default package for tabbyapi module to be broken pretty much, it needs for cuda to be enabled for it to work. How do other modules do this? Do they set the default package to a cuda enabled variant somehow or do they expect the user to enable cuda themselves?

16:44:52
@kaya:catnip.eekaya 𖤐I tested adding the PR as a patch to flash-attn, it indeed no longer OOMs which is nice, but it also doesn't build, seems to get infinitely stuck on building 16:46:56
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Yes the user should enable CUDA. Generally going through variants (like pkgsCuda) shouldn’t be permissible in-tree. You can add an assertion to the module to require sure cuda support is configured.19:19:14
@kaya:catnip.eekaya 𖤐Hm okay, thank you. I guess assertion with a specific message is better than nothing19:21:29
10 Mar 2026
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)13.2 is out 🫩 https://developer.download.nvidia.com/compute/cuda/redist/03:35:22
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) danielrf Orin is supported by 13.2/JP7: https://developer.nvidia.com/blog/cuda-13-2-introduces-enhanced-cuda-tile-support-and-new-python-features/#embedded_devices 06:10:08
@glepage:matrix.orgGaétan Lepage I got you connor (burnt/out) (UTC-8)
https://github.com/NixOS/nixpkgs/pull/498523
11:52:20
@glepage:matrix.orgGaétan Lepage We do have libcublasmp. Is this doc outdated? https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/cuda-modules/README.md#distinguished-packages 12:31:03
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Ah yep it’s outdated, I packages nvshmem: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/cuda-modules/packages/libnvshmem.nix15:28:13
@glepage:matrix.orgGaétan Lepage connor (burnt/out) (UTC-8) if I want to bump libcublasmp (to 0.7.x) for example, how do I know which cudaPackage_X should be affected? 17:36:23
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)A very deep reading of the changelog, package contents changes, and thorough rebuilds and runtime verification for consumers17:38:14
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Yet another reason we need test suites for downstream packages which exercise those libraries — relying on NVIDIA’s samples (if they’re even available) isn’t sufficient because we care about whether consumers break17:39:46
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)All of the assertions I added to the packages were the result of a ton of reading and gleaning meaning through changelogs and actual package contents changes17:40:16
@glepage:matrix.orgGaétan LepageSounds like a ton of fun :')17:47:23
@cameron-matrix:matrix.orgCameron Barker joined the room.18:18:26
11 Mar 2026
@justbrowsing:matrix.orgKevin Mittman (jetlagged/UTC+8)Redacted or Malformed Event01:54:11
@glepage:matrix.orgGaétan Lepage connor (burnt/out) (UTC-8) would you agree with a 12.8 -> 12.9 global bump before messing around with 13.0? 11:05:21
@ctheune:matrix.flyingcircus.ioChristian Theune changed their display name from Theuni to Christian Theune.14:13:00
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Sure! I remember some weird breakages a while back when I had wanted to bump immediately after 12.9 became available, but hopefully they’re all resolved by now :)16:08:54
@glepage:matrix.orgGaétan Lepage https://github.com/NixOS/nixpkgs/pull/498861 16:43:46
@glepage:matrix.orgGaétan Lepage

connor (burnt/out) (UTC-8)
About https://github.com/NixOS/nixpkgs/pull/498681, I plan to build torch and vllm. If this works fine, I will merge it.
With the CUDA PRs on the way, I won't have the capacity to exhaustively test all of them.

No objection on your side?

23:37:24
@glepage:matrix.orgGaétan Lepage (same reasonning for https://github.com/NixOS/nixpkgs/pull/498678#issuecomment-4035473707). 23:39:46
@glepage:matrix.orgGaétan Lepage * (same reasonning for https://github.com/NixOS/nixpkgs/pull/498678). 23:39:52
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Sounds good! I’ll leave a comment on them23:51:04
@glepage:matrix.orgGaétan Lepage I'm testing the CUDA bump more thoroughly though.
~1.3k rebuilds left (out of 1.8k)
23:53:19
@glepage:matrix.orgGaétan Lepage *

connor (burnt/out) (UTC-8)
About https://github.com/NixOS/nixpkgs/pull/498681, I plan to build torch and vllm. If this works fine, I will merge it.
With all the CUDA PRs in the queue, I won't have the capacity to exhaustively test all of them.

No objection on your side?

23:54:04
4 Aug 2022
@winterqt:nixos.devWinter (she/her) joined the room.03:26:42
@winterqt:nixos.devWinter (she/her)(hi, just came here to read + respond to this.)03:28:52
@tpw_rules:matrix.orgtpw_ruleshey. i had previously sympathzied with samuela and like i said before had some of the same frustrations. i just edited my github comment to add "[CUDA] packages are universally complicated, fragile to package, and critical to daily operations. Nix being able to manage them is unbelievably helpful to those of us who work with them regularly, even if support is downgraded to only having an expectation of function on stable branches."03:29:14
@winterqt:nixos.devWinter (she/her)
In reply to @tpw_rules:matrix.org
i'm mildly peeved about a recent merging of something i maintain where i'm pretty sure the merger does not own the expensive hardware required to properly test the package. i don't think it broke anything but i was given precisely 45 minutes to see the notification before somebody merged it
ugh, 45 minutes? that's... not great. not to air dirty laundry but did you do what samuela did in the wandb PR and at least say that that wasn't a great thing to do? (not sure how else to word that, you get what i mean)
03:30:23

Show newer messages


Back to Room ListRoom Version: 9