full log:

<ixpkgs-review -- pr 460701 --extra-nixpkgs-config '{ cudaSupport = true; allowUnfree = true; }'

================================================================================
PR #460701: python3Packages.vllm: enhance CLI UX
================================================================================
Author: daniel-fahey
Branch: daniel-fahey:fix/python3Packages.vllm -> NixOS:master
State: open

Description:
----------------------------------------
Replaces the unofficial and deprecated `pynvml` library with the official `nvidia-ml-py` in vLLM (maintainers: @happysalada, @CertainLach) and Flashinfer (one of its dependencies, maintainer: @breakds).

Using `nvidia-ml-py` with its [patch for Nix](https://github.com/NixOS/nixpkgs/blob/b6a8526db03f735b89dd5ff348f53f752e7ddc8e/pkgs/development/python-modules/nvidia-ml-py/0001-locate-libnvidia-ml.so.1-on-NixOS.patch) means
1. Not needing to set `LD_LIBRARY_PATH=/run/opengl-driver/lib` before using `vllm` commands to avoid runtime errors with CUDA
2. Avoiding the `FutureWarning`

Both can be [a pain point for new users](https://github.com/NixOS/nixpkgs/issues/450190).

At the time of writing:

git ls-remote https://github.com/NixOS/nixpkgs nixos-unstable

b6a8526db03f735b89dd5ff348f53f752e7ddc8e        refs/heads/nixos-unstable

Before 1:
```bash
nix-shell --pure \
--arg config '{ allowUnfree = true; cudaSupport = true; }' \
--include nixpkgs=https://github.com/NixOS/nix

... (truncated)
----------------------------------------

Files changed (2 files):
  - pkgs/development/python-modules/flashinfer/default.nix
  - pkgs/development/python-modules/vllm/default.nix
================================================================================

No GitHub token provided via GITHUB_TOKEN variable.
Falling back to local evaluation.
Tip: Install the `gh` command line tool and run `gh auth login` to authenticate.
$ git -c fetch.prune=false fetch --no-tags --force https://github.com/NixOS/nixpkgs 89259bacf0ad1dd05272a2023ec14d247a0b7f20:refs/nixpkgs-review/0
remote: Enumerating objects: 42, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 26 (delta 21), reused 13 (delta 9), pack-reused 0 (from 0)
Unpacking objects: 100% (26/26), 3.52 KiB | 45.00 KiB/s, done.
From https://github.com/NixOS/nixpkgs
 + 5738ba8af0aa...89259bacf0ad 89259bacf0ad1dd05272a2023ec14d247a0b7f20 -> refs/nixpkgs-review/0  (forced update)
$ git worktree add /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs 32456a1a360d41b50397ce29b4dd92a261b52ff9
Preparing worktree (detached HEAD 32456a1a360d)
Updating files: 100% (49740/49740), done.
HEAD is now at 32456a1a360d zeno: 2.0.17 -> 2.0.18 (#460698)
Local evaluation for computing rebuilds
$ nix-env --extra-experimental-features no-url-literals --option system x86_64-linux -f <nixpkgs> --nix-path nixpkgs=/mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs nixpkgs-overlays=/tmp/tmpwslfu0b_ -qaP --xml --out-path --show-trace --no-allow-import-from-derivation
$ git checkout 89259bacf0ad1dd05272a2023ec14d247a0b7f20
Previous HEAD position was 32456a1a360d zeno: 2.0.17 -> 2.0.18 (#460698)
HEAD is now at 89259bacf0ad Merge 66fa9f1d2aed43cdc2803f17c22eb8323df99c39 into 32456a1a360d41b50397ce29b4dd92a261b52ff9
$ nix-env --extra-experimental-features no-url-literals --option system x86_64-linux -f <nixpkgs> --nix-path nixpkgs=/mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs nixpkgs-overlays=/tmp/tmpwslfu0b_ -qaP --xml --out-path --show-trace --no-allow-import-from-derivation --meta
error:
       … while querying the derivation named 'python3.12-kserve-0.16.0'

       … while evaluating the attribute 'out.outPath'
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/lib/customisation.nix:411:13:
          410|               drv.${outputName}.drvPath;
          411|             outPath =
             |             ^
          412|               assert condition;

       … while evaluating the attribute 'out.outPath'
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/lib/customisation.nix:411:13:
          410|               drv.${outputName}.drvPath;
          411|             outPath =
             |             ^
          412|               assert condition;

       … while calling the 'getAttr' builtin
         at <nix/derivation-internal.nix>:50:17:
           49|     value = commonAttrs // {
           50|       outPath = builtins.getAttr outputName strict;
             |                 ^
           51|       drvPath = strict.drvPath;

       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:37:12:
           36|
           37|   strict = derivationStrict drvAttrs;
             |            ^
           38|

       … while evaluating derivation 'python3.12-kserve-0.16.0'
         whose name attribute is located at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:544:13

       … while evaluating attribute 'nativeBuildInputs' of derivation 'python3.12-kserve-0.16.0'
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:593:13:
          592|             depsBuildBuild = elemAt (elemAt dependencies 0) 0;
          593|             nativeBuildInputs = elemAt (elemAt dependencies 0) 1;
             |             ^
          594|             depsBuildTarget = elemAt (elemAt dependencies 0) 2;

       … while evaluating the attribute 'out.outPath'
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/lib/customisation.nix:411:13:
          410|               drv.${outputName}.drvPath;
          411|             outPath =
             |             ^
          412|               assert condition;

       … in the condition of the assert statement
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/lib/customisation.nix:412:15:
          411|             outPath =
          412|               assert condition;
             |               ^
          413|               drv.${outputName}.outPath;

       … from call site
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:836:18:
          835|       };
          836|       validity = checkMeta.assertValidity { inherit meta attrs; };
             |                  ^
          837|

       … while calling 'assertValidity'
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:739:5:
          738|   assertValidity =
          739|     { meta, attrs }:
             |     ^
          740|     let

       … while evaluating a branch condition
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:744:5:
          743|     in
          744|     if validity ? handled then
             |     ^
          745|       validity

       … from call site
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:741:18:
          740|     let
          741|       validity = checkValidity attrs;
             |                  ^
          742|       inherit (validity) valid;

       … while calling anonymous lambda
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:498:5:
          497|     in
          498|     attrs:
             |     ^
          499|     # Check meta attribute types first, to make sure it is always called even when there are other issues

       … while evaluating a branch condition
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:504:5:
          503|     in
          504|     if res != [ ] then
             |     ^
          505|       {

       … from call site
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:502:13:
          501|     let
          502|       res = checkMeta (attrs.meta or { });
             |             ^
          503|     in

       … while calling 'checkMeta'
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:471:5:
          470|   checkMeta =
          471|     meta:
             |     ^
          472|     optionals config.checkMeta (concatMap (attr: checkMetaAttr attr meta.${attr}) (attrNames meta));

       … from call site
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:472:5:
          471|     meta:
          472|     optionals config.checkMeta (concatMap (attr: checkMetaAttr attr meta.${attr}) (attrNames meta));
             |     ^
          473|

       … while calling 'optionals'
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/lib/lists.nix:811:21:
          810|   */
          811|   optionals = cond: elems: if cond then elems else [ ];
             |                     ^
          812|

       … while calling the 'concatMap' builtin
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:472:33:
          471|     meta:
          472|     optionals config.checkMeta (concatMap (attr: checkMetaAttr attr meta.${attr}) (attrNames meta));
             |                                 ^
          473|

       … while calling anonymous lambda
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:472:44:
          471|     meta:
          472|     optionals config.checkMeta (concatMap (attr: checkMetaAttr attr meta.${attr}) (attrNames meta));
             |                                            ^
          473|

       … from call site
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:472:50:
          471|     meta:
          472|     optionals config.checkMeta (concatMap (attr: checkMetaAttr attr meta.${attr}) (attrNames meta));
             |                                                  ^
          473|

       … while calling anonymous lambda
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:454:8:
          453|     in
          454|     k: v:
             |        ^
          455|     if metaTypes ? ${k} then

       … while evaluating a branch condition
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:456:7:
          455|     if metaTypes ? ${k} then
          456|       if metaTypes'.${k} v then
             |       ^
          457|         [ ]

       … from call site
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/check-meta.nix:456:10:
          455|     if metaTypes ? ${k} then
          456|       if metaTypes'.${k} v then
             |          ^
          457|         [ ]

       … while calling anonymous lambda
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/meta-types.nix:84:43:
           83|         # listOf<any> can be optimised to just isList
           84|         if t == self.any then isList else v: isList v && all verify v;
             |                                           ^
           85|     };

       … in the right operand of the AND (&&) operator
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/meta-types.nix:84:55:
           83|         # listOf<any> can be optimised to just isList
           84|         if t == self.any then isList else v: isList v && all verify v;
             |                                                       ^
           85|     };

       … while calling the 'all' builtin
         at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/stdenv/generic/meta-types.nix:84:58:
           83|         # listOf<any> can be optimised to just isList
           84|         if t == self.any then isList else v: isList v && all verify v;
             |                                                          ^
           85|     };

       … while calling the 'isAttrs' builtin

       error: undefined variable 'CertainLach'
       at /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs/pkgs/development/python-modules/vllm/default.nix:511:7:
          510|     maintainers = with lib.maintainers; [
          511|       CertainLach
             |       ^
          512|       daniel-fahey
https://github.com/NixOS/nixpkgs/pull/460701 failed to build: Failed to list packages: nix-env failed with exit code 1
$ git worktree remove -f /mnt/weka/home/ari/.cache/nixpkgs-review/pr-460701/nixpkgs

In reply to @arilotter:matrix.org
image.png

Since it seems you're in the mood to watch computer go brrrrr, I use this command to monitor (and record stats) during compiles

nix-shell -p dool --run 'dool -tcms --swap --top-mem --top-cpu --display --output ~/vllm-build-stats.csv 10'

oh and if you're building JAX, that must mean you're not using Flox's new cache?

https://discourse.nixos.org/t/nix-flox-nvidia-opening-up-cuda-redistribution-on-nix/69189

extra-trusted-substituters = https://cache.flox.dev
extra-trusted-public-keys = flox-cache-public-1:7F4OyH7ZCnFhcze3fJdfyXYLQw/aV7GEed86nQ7IsOs=

Oh, forgot, the command you'll want to post to the PR as a comment, use

nixpkgs-review pr 460701 --extra-nixpkgs-config '{ cudaSupport = true; allowUnfree = true;}' --post-result
``` thanks again!

Oh, forgot, the command you'll want to post to the PR as a comment, use

nixpkgs-review pr 460701 --extra-nixpkgs-config '{ cudaSupport = true; allowUnfree = true;}' --post-result
```
thanks again!