!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

289 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda57 Servers

Load older messages


SenderMessageTime
18 Jan 2026
@glepage:matrix.orgGaétan Lepage Anyone could give a second opinion on https://github.com/NixOS/nixpkgs/pull/474676 please? 09:48:06
@ss:someonex.netSomeoneSerge (back on matrix)This particular regression could've been avoided with autopatchelf. Maybe switch to that?12:08:40
@glepage:matrix.orgGaétan Lepage Just adding autoPatchelfHook you mean? 22:29:11
@ss:someonex.netSomeoneSerge (back on matrix)And removing --set-rpath23:40:44
20 Jan 2026
@kaya:catnip.eekaya 𖤐 Gaétan Lepage: Thank you for the through PR review, much appreciated! 19:03:45
@glepage:matrix.orgGaétan LepageYou're welcome! I'll probably wait for a second committer to double check, but we should be able to merge it soon :)19:26:32
21 Jan 2026
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) Gaétan Lepageif you haven’t already, can you link any of the additional work you’ve had to do for the OpenCV version bump in the PR description? I remember yesterday you had mentioned needing to fix protobuf, but that might have been about ONNX Runtime. 17:20:11
@glepage:matrix.orgGaétan Lepage Indeed, this was for protobuf. The opencv effort is all contained in https://github.com/NixOS/nixpkgs/pull/479136 21:33:06
@glepage:matrix.orgGaétan Lepage The protobuf patching related to onnxruntime was merged into staging.
-> https://github.com/NixOS/nixpkgs/pull/480716
21:33:34
22 Jan 2026
@glepage:matrix.orgGaétan Lepage

Sadness.

I had to package the 12.8 version of cuda-bindings, because we're on cuda 12.8.
However, it dlopens libcuda.so and the driver (on my system at least) tells it that it uses cuda 13.1.
Hence, it errors out :/

https://github.com/NixOS/nixpkgs/pull/482638

(cuda-bindings is supposed to be required in the new torch update, not sure how critical it is though)

12:06:42
@justbrowsing:matrix.orgKevin Mittman (UTC-8) Hmm a bit confused about that issue, as nvidia-smi shows the maximum CUDA version supported and with "backwards compat" should always be able to use a newer NVIDIA driver than a CUDA Toolkit was released with 20:14:30
@glepage:matrix.orgGaétan Lepage The version detection logic happens here and errors out here. 20:48:39
@glepage:matrix.orgGaétan Lepage

Here's the stack trace when I re-enable the failing test:

______________________ ERROR collecting test_nvjitlink.py ______________________
tests/test_nvjitlink.py:62: in <module>
    not check_nvjitlink_usable(), reason="nvJitLink not usable, maybe not installed or too old (<12.3)"
        ^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_nvjitlink.py:58: in check_nvjitlink_usable
    return inner_nvjitlink._inspect_function_pointer("__nvJitLinkVersion") != 0
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cuda/bindings/_internal/nvjitlink.pyx:254: in cuda.bindings._internal.nvjitlink._inspect_function_pointer
    ???
cuda/bindings/_internal/nvjitlink.pyx:257: in cuda.bindings._internal.nvjitlink._inspect_function_pointer
    ???
cuda/bindings/_internal/nvjitlink.pyx:205: in cuda.bindings._internal.nvjitlink._inspect_function_pointers
    ???
cuda/bindings/_internal/nvjitlink.pyx:99: in cuda.bindings._internal.nvjitlink._check_or_init_nvjitlink
    ???
cuda/bindings/_internal/nvjitlink.pyx:57: in cuda.bindings._internal.nvjitlink.load_library
    ???
cuda/bindings/_internal/utils.pyx:137: in cuda.bindings._internal.utils.get_nvjitlink_dso_version_suffix
    ???
E   cuda.bindings._internal.utils.NotSupportedError: only CUDA 12 driver is supported
------------------------------- Captured stdout --------------------------------
+++++++++++++++ 13010
22:27:19
@glepage:matrix.orgGaétan Lepage The code block mentionned above returns 13010 which fails the 12000 <= driver_ver < 13000 check in cuda/bindings/_internal/utils.pyx. 22:28:37
23 Jan 2026
@glepage:matrix.orgGaétan Lepage Weird regression following the merge of staging-next... 10:42:06
@arilotter:matrix.orgAri Lotter

hm i'm getting an infinite hang on pytestCheckPhase for python3.12-keras3.13.1 since the 3.13.1 update 🤔 https://github.com/NixOS/nixpkgs/commit/73806a397d9f9360f331ed98202e5cecc1230548
building python3.12-keras-3.13.1 (pytestCheckPhase): keras/src/trainers/trainer_test.py::TestTrainer::test_fit_with_data_adapter_py_dataset_infinite_cw PASSED [ 94%]
hang happens every time

nix-top shows

  UID     PID    PPID STIME     TIME COMMAND
30001 3422767 3422639 15:44 00:00:00 bash -e /nix/store/l622p70vy8k5sh7y5wizi5f2mic6ynpg-source-stdenv.sh /nix/store/shkw4qm9qcw5sc5n1k5jznc83ny02r39-default-builder.sh
30001 3425989 3422767 15:45 00:17:09 /nix/store/iizfzmqjdfvbyx82gqw92glb5zl6m4my-python3-3.12.12/bin/python3.12 -m pytest --ignore-glob=keras/src/layers/preprocessing/data_layer_test.py --ignore-glob=keras/src/layers/preprocessing/image_preprocessing/resizing_test.py --ignore-glob=keras/src/layers/preprocessing/rescaling_test.py --ignore-glob=keras/src/trainers/data_adapters/grain_dataset_adapter_test.py --ignore-glob=keras/src/export/onnx_test.py --ignore-glob=integration_tests/dataset_tests --ignore-glob=keras/src/applications/applications_test.py --ignore-glob=integration_tests/jax_custom_fit_test.py --ignore-glob=integration_tests/tf_distribute_training_test.py --ignore-glob=integration_tests/torch_custom_fit_test.py --ignore-glob=integration_tests/torch_workflow_test.py --ignore-glob=keras/src/backend/tensorflow/saved_model_test.py -k not (test_fit_with_data_adapter_grain_dataloader) and not (test_fit_with_data_adapter_grain_datast) and not (test_fit_with_data_adapter_grain_datast_with_len) and not (test_image_dataset_from_directory_binary_grain) and not (test_image_dataset_from_directory_color_modes_grain) and not (test_image_dataset_from_directory_crop_to_aspect_ratio_grain) and not (test_image_dataset_from_directory_follow_links_grain) and not (test_image_dataset_from_directory_manual_labels_grain) and not (test_image_dataset_from_directory_multiclass_grain) and not (test_image_dataset_from_directory_no_labels_grain) and not (test_image_dataset_from_directory_not_batched_grain) and not (test_image_dataset_from_directory_pad_to_aspect_ratio_grain) and not (test_image_dataset_from_directory_shuffle_grain) and not (test_image_dataset_from_directory_validation_split_grain) and not (test_sample_count_grain) and not (test_text_dataset_from_directory_binary_grain) and not (test_text_dataset_from_directory_follow_links_grain) and not (test_text_dataset_from_directory_manual_labels_grain) and not (test_text_dataset_from_directory_multiclass_grain) and not (test_text_dataset_from_directory_not_batched_grain) and not (test_text_dataset_from_directory_standalone_grain) and not (test_text_dataset_from_directory_validation_split_grain) and not (test_keras_imports) and not (test_reloading_default_saved_model)
30001 3797431 3425989 15:59 00:00:00 /nix/store/iizfzmqjdfvbyx82gqw92glb5zl6m4my-python3-3.12.12/bin/python3.12 -m pytest --ignore-glob=keras/src/layers/preprocessing/data_layer_test.py --ignore-glob=keras/src/layers/preprocessing/image_preprocessing/resizing_test.py --ignore-glob=keras/src/layers/preprocessing/rescaling_test.py --ignore-glob=keras/src/trainers/data_adapters/grain_dataset_adapter_test.py --ignore-glob=keras/src/export/onnx_test.py --ignore-glob=integration_tests/dataset_tests --ignore-glob=keras/src/applications/applications_test.py --ignore-glob=integration_tests/jax_custom_fit_test.py --ignore-glob=integration_tests/tf_distribute_training_test.py --ignore-glob=integration_tests/torch_custom_fit_test.py --ignore-glob=integration_tests/torch_workflow_test.py --ignore-glob=keras/src/backend/tensorflow/saved_model_test.py -k not (test_fit_with_data_adapter_grain_dataloader) and not (test_fit_with_data_adapter_grain_datast) and not (test_fit_with_data_adapter_grain_datast_with_len) and not (test_image_dataset_from_directory_binary_grain) and not (test_image_dataset_from_directory_color_modes_grain) and not (test_image_dataset_from_directory_crop_to_aspect_ratio_grain) and not (test_image_dataset_from_directory_follow_links_grain) and not (test_image_dataset_from_directory_manual_labels_grain) and not (test_image_dataset_from_directory_multiclass_grain) and not (test_image_dataset_from_directory_no_labels_grain) and not (test_image_dataset_from_directory_not_batched_grain) and not (test_image_dataset_from_directory_pad_to_aspect_ratio_grain) and not (test_image_dataset_from_directory_shuffle_grain) and not (test_image_dataset_from_directory_validation_split_grain) and not (test_sample_count_grain) and not (test_text_dataset_from_directory_binary_grain) and not (test_text_dataset_from_directory_follow_links_grain) and not (test_text_dataset_from_directory_manual_labels_grain) and not (test_text_dataset_from_directory_multiclass_grain) and not (test_text_dataset_from_directory_not_batched_grain) and not (test_text_dataset_from_directory_standalone_grain) and not (test_text_dataset_from_directory_validation_split_grain) and not (test_keras_imports) and not (test_reloading_default_saved_model)
30001 3797433 3425989 15:59 00:00:00 /nix/store/iizfzmqjdfvbyx82gqw92glb5zl6m4my-python3-3.12.12/bin/python3.12 -m pytest --ignore-glob=keras/src/layers/preprocessing/data_layer_test.py --ignore-glob=keras/src/layers/preprocessing/image_preprocessing/resizing_test.py --ignore-glob=keras/src/layers/preprocessing/rescaling_test.py --ignore-glob=keras/src/trainers/data_adapters/grain_dataset_adapter_test.py --ignore-glob=keras/src/export/onnx_test.py --ignore-glob=integration_tests/dataset_tests --ignore-glob=keras/src/applications/applications_test.py --ignore-glob=integration_tests/jax_custom_fit_test.py --ignore-glob=integration_tests/tf_distribute_training_test.py --ignore-glob=integration_tests/torch_custom_fit_test.py --ignore-glob=integration_tests/torch_workflow_test.py --ignore-glob=keras/src/backend/tensorflow/saved_model_test.py -k not (test_fit_with_data_adapter_grain_dataloader) and not (test_fit_with_data_adapter_grain_datast) and not (test_fit_with_data_adapter_grain_datast_with_len) and not (test_image_dataset_from_directory_binary_grain) and not (test_image_dataset_from_directory_color_modes_grain) and not (test_image_dataset_from_directory_crop_to_aspect_ratio_grain) and not (test_image_dataset_from_directory_follow_links_grain) and not (test_image_dataset_from_directory_manual_labels_grain) and not (test_image_dataset_from_directory_multiclass_grain) and not (test_image_dataset_from_directory_no_labels_grain) and not (test_image_dataset_from_directory_not_batched_grain) and not (test_image_dataset_from_directory_pad_to_aspect_ratio_grain) and not (test_image_dataset_from_directory_shuffle_grain) and not (test_image_dataset_from_directory_validation_split_grain) and not (test_sample_count_grain) and not (test_text_dataset_from_directory_binary_grain) and not (test_text_dataset_from_directory_follow_links_grain) and not (test_text_dataset_from_directory_manual_labels_grain) and not (test_text_dataset_from_directory_multiclass_grain) and not (test_text_dataset_from_directory_not_batched_grain) and not (test_text_dataset_from_directory_standalone_grain) and not (test_text_dataset_from_directory_validation_split_grain) and not (test_keras_imports) and not (test_reloading_default_saved_model)
16:42:06
@arilotter:matrix.orgAri Lotterah lol still happens on 3.13.0 - problem is that it's not binary cached and i think just hangs forever on this system (that weird intel server i have) 😅17:48:54
@arilotter:matrix.orgAri Lotter does nixpkgs have a mechanism for marking a package's checks as broken on a specific CPU generation? 18:01:03
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Nix barely knows the platform; it doesn’t know what generation the hardware is (or at least doesn’t make it available)19:42:59
@apyh:matrix.orgapyhyeah, makes sense. 's frustrating, means i have to maintain some out of tree list of overrides for checks to disable for specific packages 20:06:37
@sporeray:matrix.orgRobbie BuxtonHave you tried nixbuild.net?20:11:49
@sporeray:matrix.orgRobbie Buxton I use it and it’s fantastic 20:11:58
@sporeray:matrix.orgRobbie Buxton Never had an issue building anything there 20:12:43
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)There’s some support for architecture feature sets but you’ll have to rebuild everything20:30:36
@apyh:matrix.orgapyhyeah, i use garnix.io for CI stuff, this is more frustrating that i have a ton of compute available to me & i have to reserve a gpu node anyways to run the stuff I'm testing, but I can't build it on the same box :p20:47:22
@apyh:matrix.orgapyhprobably something to report w the upstream package though :)20:47:33
4 Aug 2022
@winterqt:nixos.devWinter (she/her) joined the room.03:26:42
@winterqt:nixos.devWinter (she/her)(hi, just came here to read + respond to this.)03:28:52
@tpw_rules:matrix.orgtpw_ruleshey. i had previously sympathzied with samuela and like i said before had some of the same frustrations. i just edited my github comment to add "[CUDA] packages are universally complicated, fragile to package, and critical to daily operations. Nix being able to manage them is unbelievably helpful to those of us who work with them regularly, even if support is downgraded to only having an expectation of function on stable branches."03:29:14
@winterqt:nixos.devWinter (she/her)
In reply to @tpw_rules:matrix.org
i'm mildly peeved about a recent merging of something i maintain where i'm pretty sure the merger does not own the expensive hardware required to properly test the package. i don't think it broke anything but i was given precisely 45 minutes to see the notification before somebody merged it
ugh, 45 minutes? that's... not great. not to air dirty laundry but did you do what samuela did in the wandb PR and at least say that that wasn't a great thing to do? (not sure how else to word that, you get what i mean)
03:30:23

Show newer messages


Back to Room ListRoom Version: 9