| 12 Nov 2025 |
 Ari Lotter | just throwing nixpkgs-review back on for convenience lol | 16:50:29 |
 Daniel Fahey | I've never really looked in the internals of it, just a bit worried it might be changing something idk | 16:51:09 |
| Ari Lotter | ok, lemme... | 16:51:32 |
| Daniel Fahey | (probably an unwarranted concern)
just interested to see what happens when JAX is built in isolation | 16:52:40 |
| Daniel Fahey | Still, really similar experience that I had in https://github.com/NixOS/nixpkgs/issues/445824 | 16:53:05 |
| Ari Lotter | ok gonna try
ari@dell-h100-nj4-2:~/src/nixpkgs$ git fetch origin pull/460701/head:pr-460701
From https://github.com/NixOS/nixpkgs
* [new ref] refs/pull/460701/head -> pr-460701
ari@dell-h100-nj4-2:~/src/nixpkgs$ git checkout pr-460701
nix buildSwitched to branch 'pr-460701'
ari@dell-h100-nj4-2:~/src/nixpkgs$ nix-build -A "python312Packages.jax" --arg config '{ cudaSupport = true; allowUnfree = true; }'
| 16:56:02 |
| Ari Lotter | taht should work, yea? | 16:56:04 |
| Ari Lotter | * ok gonna try
ari@dell-h100-nj4-2:~/src/nixpkgs$ git fetch origin pull/460701/head:pr-460701
From https://github.com/NixOS/nixpkgs
* [new ref] refs/pull/460701/head -> pr-460701
ari@dell-h100-nj4-2:~/src/nixpkgs$ git checkout pr-460701
Switched to branch 'pr-460701'
ari@dell-h100-nj4-2:~/src/nixpkgs$ nix-build -A "python312Packages.jax" --arg config '{ cudaSupport = true; allowUnfree = true; }'
| 16:56:10 |
| Daniel Fahey | Yeah, I think so. I tend to use this one-liner, to not have to use git worktrees etc.:
nix-build https://github.com/daniel-fahey/nixpkgs/archive/fix/python3Packages.vllm.tar.gz \
--pure --arg config '{ allowUnfree = true; cudaSupport = true; }' \
--attr python312Packages.jax
| 16:58:59 |
| Daniel Fahey | I have noticed that the pytestFlags are hard-coded and aren't affected by -j/--max-jobs:
https://github.com/NixOS/nixpkgs/blob/5d2aef6a2766b8d97f520563aace1c9b7f538051/pkgs/development/python-modules/jax/default.nix#L96-L99
| 17:06:36 |
| Daniel Fahey | so you'd have to use an overlay to change it, like this:
nix-build https://github.com/daniel-fahey/nixpkgs/archive/fix/python3Packages.vllm.tar.gz --pure --arg config '{ allowUnfree = true; cudaSupport = true; }' --arg overlays '[
(self: super: {
python312Packages = super.python312Packages.override {
overrides = pythonSelf: pythonSuper: {
jax = pythonSuper.jax.overrideAttrs (oldAttrs: {
pytestFlags = [
"--numprocesses=2"
"-Wignore::DeprecationWarning"
];
});
};
};
})
]' --attr python312Packages.jax
could probably do with refactoring the derivation? Gaétan Lepage (sorry for bugging you again)
| 17:11:13 |
| Ari Lotter | ahaaa, errors! | 17:14:46 |
| Ari Lotter | yep | 17:15:00 |
| Ari Lotter | [gw3] node down: Not properly terminated F replacing crashed worker gw3 | 17:15:01 |
| Ari Lotter | fails :) | 17:15:03 |
| Ari Lotter | running again to hopefully get a real message.. | 17:15:11 |
| Daniel Fahey | any luck with this approach? | 17:22:28 |
| Ari Lotter | ah will try that next | 17:22:41 |
| Ari Lotter | should i just import from hydra? | 17:23:18 |
| Ari Lotter | then move on? | 17:23:20 |
| Daniel Fahey | Up to you, I'm in your debt already, I thought you were having fun messing about and learning | 17:53:34 |
| Ari Lotter | oh yeah i meant like, to run review | 17:56:26 |
| Ari Lotter | how big is this closure lmao | 17:56:30 |
| Ari Lotter | im at 3gb downloaded so far | 17:56:34 |
| Ari Lotter | oh there we go | 17:56:40 |
| Ari Lotter | ok will try the review again heh | 17:57:13 |
| Ari Lotter | huh. nixpkgs at b3d51a0365f6695e7dd5cdf3e180604530ed33b4 (based off nixos-unstable)` seems to have something broken with torch / cuda / cudnn...
torch 2.8 worked fine, but 2.9 with my same application code throws
[cuda:0]: cuDNN Frontend error: No valid engine configs for Matmul_MUL_ADD_Reduction_SUB_EXP_Reduction_LOG_ADD_DIV_Matmul_
{"engineId":2,"smVersion":900,"knobChoices":{"CUDNN_KNOB_TYPE_KERNEL_CFG":7}}
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: Could not open libnvrtc at: this->libnvrtc == nullptr
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: compiler.load()
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: rtk(kernelNumRunning)->loadDLL()
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: status == CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: ptr->isSupported()
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: finalize_internal()
{"engineId":1,"smVersion":900,"knobChoices":{"CUDNN_KNOB_TYPE_KERNEL_CFG":3}}
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: Could not open libnvrtc at: this->libnvrtc == nullptr
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: compiler.load()
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: rtk(kernelNumRunning)->loadDLL()
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: status == CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: ptr->isSupported()
Warning: CUDNN_STATUS_NOT_SUPPORTED_RUNTIME_PREREQUISITE_MISSING; Reason: finalize_internal()
| 17:58:30 |
| Ari Lotter | missing nvrtc i guess.. | 17:58:40 |
| Ari Lotter | gonna compare against my weird 2.9.0-nightly setup i was using before.. | 18:00:37 |
| Daniel Fahey | Would need a reprex I think with the smallest bit of your application code that can trigger this | 18:40:46 |
