ATTEMPTS.md - Chronicles all 8 attempts with re-evaluation using current knowledge:

1. Nixpkgs torch (no CUDA)
2. Build from source (OOM)
3. Nix Python + pip (glibc conflicts)
4. System Python + pip (works but not reproducible)
5. fetchurl wheels (incomplete)
6. Copy venv (incomplete) 7. buildPythonPackage test (learning)
7. Hybrid solution (SUCCESS)

EXPLANATION.md - Explains WHY the solution works:

• The glibc problem and how we solved it
• Why Nix Python + pip wheels is the right approach
• How makeLibraryPath simplifies library management
• The trade-off between purity and practicality

Hi folks, I've been working on compiling XLA in nix with CUDA support, but I'm running into this issue of the current nixpkgs glibc containing symbols (incl. cospif, rsqrtf, sinpi, cospi, rsqrt) that conflict with CUDA defined symbols:

glibc 2.42 (via __MATHCALL → __MATHDECL_1_IMPL):
extern float sinpif(float __x) noexcept(true); // __THROW → noexcept(true) in C++

CUDA (crt/math_functions.h):
extern host device float sinpif(float x); // no noexcept

has anyone else encountered this? if so how did you handle it?

This PR was harder to finish than I expected. It's now ready and fixes a bunch of cudaSupport package builds.
https://github.com/NixOS/nixpkgs/pull/495151

(waiting for reviews)

[FYI: vllm]

vllm 0.16.0 update merged.
https://github.com/NixOS/nixpkgs/pull/490175

Changelog: https://github.com/vllm-project/vllm/releases/tag/v0.16.0

In reply to @connorbaker:matrix.org
https://github.com/NixOS/nixpkgs/blob/3bb5f20c47dcfcab9acb3be810f42ca1261b49e2/pkgs/development/cuda-modules/packages/cuda_nvcc.nix#L167

Im currently in process of upstreaming the nixos module for tabbapi https://github.com/NixOS/nixpkgs/pull/498281
Does anyone know how i would go about setting the default package? In my nixos config i use the module like this right now, i always override the package:

    services.tabbyapi = {
      enable = true;
      package = pkgs.pkgsCuda.tabbyapi;
    };

I feel like it might be bad to have the default package for tabbyapi module to be broken pretty much, it needs for cuda to be enabled for it to work. How do other modules do this? Do they set the default package to a cuda enabled variant somehow or do they expect the user to enable cuda themselves?