!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

281 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda58 Servers

Load older messages


SenderMessageTime
12 Nov 2025
@arilotter:matrix.orgAri Lotter i'm of the belief that being mean to anything, even inanimate objects like a rock or an LLM or a piece of software someone else wrote, is bad for your "soul" x3 16:42:42
@arilotter:matrix.orgAri Lotteranyways16:42:44
@daniel-fahey:matrix.orgDaniel FaheyYou're absolutely right!16:43:17
@daniel-fahey:matrix.orgDaniel FaheyI still have a hunch it's a problem when using newer server grade Intel chips for the build16:44:24
@daniel-fahey:matrix.orgDaniel Faheynot sure how to prove that though16:44:43
@daniel-fahey:matrix.orgDaniel FaheyThanks for everything though, and the logs, nice to meet you16:47:42
@arilotter:matrix.orgAri Lotterhahaha nice to meet you too! raising ulimits to unlimited and trying again :)16:48:04
@arilotter:matrix.orgAri Lotteralso max-jobs 1 cores 2316:48:21
@arilotter:matrix.orgAri Lotter* also max-jobs 1 cores 216:48:40
@daniel-fahey:matrix.orgDaniel Fahey

i completely forgot about all that

are you building it as part of the nixpkgs-review, or separately?

16:48:51
@arilotter:matrix.orgAri Lotterjust throwing nixpkgs-review back on for convenience lol16:50:29
@daniel-fahey:matrix.orgDaniel FaheyI've never really looked in the internals of it, just a bit worried it might be changing something idk16:51:09
@arilotter:matrix.orgAri Lotterok, lemme...16:51:32
@daniel-fahey:matrix.orgDaniel Fahey(probably an unwarranted concern) just interested to see what happens when JAX is built in isolation16:52:40
@daniel-fahey:matrix.orgDaniel FaheyStill, really similar experience that I had in https://github.com/NixOS/nixpkgs/issues/44582416:53:05
@arilotter:matrix.orgAri Lotter

ok gonna try

ari@dell-h100-nj4-2:~/src/nixpkgs$ git fetch origin pull/460701/head:pr-460701
From https://github.com/NixOS/nixpkgs
 * [new ref]                   refs/pull/460701/head -> pr-460701
ari@dell-h100-nj4-2:~/src/nixpkgs$ git checkout pr-460701
nix buildSwitched to branch 'pr-460701'
ari@dell-h100-nj4-2:~/src/nixpkgs$ nix-build -A "python312Packages.jax" --arg config '{ cudaSupport = true; allowUnfree = true; }'
16:56:02
@arilotter:matrix.orgAri Lottertaht should work, yea?16:56:04
@arilotter:matrix.orgAri Lotter *

ok gonna try

ari@dell-h100-nj4-2:~/src/nixpkgs$ git fetch origin pull/460701/head:pr-460701
From https://github.com/NixOS/nixpkgs
 * [new ref]                   refs/pull/460701/head -> pr-460701
ari@dell-h100-nj4-2:~/src/nixpkgs$ git checkout pr-460701
Switched to branch 'pr-460701'
ari@dell-h100-nj4-2:~/src/nixpkgs$ nix-build -A "python312Packages.jax" --arg config '{ cudaSupport = true; allowUnfree = true; }'
16:56:10
@daniel-fahey:matrix.orgDaniel Fahey

Yeah, I think so. I tend to use this one-liner, to not have to use git worktrees etc.:

 nix-build https://github.com/daniel-fahey/nixpkgs/archive/fix/python3Packages.vllm.tar.gz \
--pure --arg config '{ allowUnfree = true; cudaSupport = true; }' \
--attr python312Packages.jax
16:58:59
@daniel-fahey:matrix.orgDaniel Fahey

I have noticed that the pytestFlags are hard-coded and aren't affected by -j/--max-jobs:

https://github.com/NixOS/nixpkgs/blob/5d2aef6a2766b8d97f520563aace1c9b7f538051/pkgs/development/python-modules/jax/default.nix#L96-L99

17:06:36
@daniel-fahey:matrix.orgDaniel Fahey

so you'd have to use an overlay to change it, like this:

nix-build https://github.com/daniel-fahey/nixpkgs/archive/fix/python3Packages.vllm.tar.gz --pure --arg config '{ allowUnfree = true; cudaSupport = true; }' --arg overlays '[
      (self: super: {
        python312Packages = super.python312Packages.override {
          overrides = pythonSelf: pythonSuper: {
            jax = pythonSuper.jax.overrideAttrs (oldAttrs: {
              pytestFlags = [
                "--numprocesses=2"
                "-Wignore::DeprecationWarning"
              ];
            });
          };
        };
      })
    ]' --attr python312Packages.jax

could probably do with refactoring the derivation? GaƩtan Lepage (sorry for bugging you again)

17:11:13
@arilotter:matrix.orgAri Lotterahaaa, errors!17:14:46
@arilotter:matrix.orgAri Lotteryep17:15:00
@arilotter:matrix.orgAri Lotter [gw3] node down: Not properly terminated F replacing crashed worker gw3 17:15:01
@arilotter:matrix.orgAri Lotterfails :)17:15:03
@arilotter:matrix.orgAri Lotterrunning again to hopefully get a real message..17:15:11
@daniel-fahey:matrix.orgDaniel Faheyany luck with this approach?17:22:28
@arilotter:matrix.orgAri Lotterah will try that next17:22:41
@arilotter:matrix.orgAri Lottershould i just import from hydra?17:23:18
@arilotter:matrix.orgAri Lotterthen move on?17:23:20

Show newer messages


Back to Room ListRoom Version: 9