!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

286 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda57 Servers

You have reached the beginning of time (for this room).


SenderMessageTime
4 Oct 2025
@daniel-fahey:matrix.orgDaniel Fahey

lon: oh you deleted, (hehe), I actually didn't even think about it and had completely forgotten nixpkgs didn't have CUDA 13 yet (https://github.com/NixOS/nixpkgs/pull/437723)

Thanks for looking into it all the same

22:29:03
@glepage:matrix.orgGaétan Lepage Daniel Fahey FYI vllm is broken on master as my PR was merged slightly too soon. 23:12:03
@glepage:matrix.orgGaétan LepageIf you have a bit of time to investigate, please go on :)23:12:12
@longregen:matrix.orglonYes, sorry I deleted because I saw your commit and is the same as mine (save for the update script! I didn't know that was a pattern people in nixpkgs used, TIL)23:13:31
@longregen:matrix.orglonthe nvidia/cutlass dependency can also be updated fwiw, with the update script 23:18:40
@longregen:matrix.orglonimage.png
Download image.png
23:18:43
@daniel-fahey:matrix.orgDaniel Faheyyeah, just started rewriting it23:23:25
@daniel-fahey:matrix.orgDaniel FaheyHow can you tell? Hydra? Got a link?23:27:58
@daniel-fahey:matrix.orgDaniel Fahey

Looks okay for me, some other problem? CUDA build?

[daniel@laptop:~/Source/nixpkgs]$ nix-build -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/b967613ed760449a73eaa73d7b69eb45e857ce1a.tar.gz --expr 'with import <nixpkgs> { }; python313Packages.vllm'
unpacking 'https://github.com/NixOS/nixpkgs/archive/b967613ed760449a73eaa73d7b69eb45e857ce1a.tar.gz' into the Git cache...
/nix/store/amncczb34wd5zingwclr3sqa6q7kahay-python3.13-vllm-0.11.0

[daniel@laptop:~/Source/nixpkgs]$ ./result/bin/vllm --help
INFO 10-05 00:44:00 [__init__.py:216] Automatically detected platform cpu.
usage: vllm [-h] [-v] {chat,complete,serve,bench,collect-env,run-batch} ...

vLLM CLI

positional arguments:
  {chat,complete,serve,bench,collect-env,run-batch}
    chat                Generate chat completions via the running API server.
    complete            Generate text completions based on the given prompt via the running API server.
    collect-env         Start collecting environment information.
    run-batch           Run batch prompts and write results to file.

options:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit

For full list:            vllm [subcommand] --help=all
For a section:            vllm [subcommand] --help=ModelConfig    (case-insensitive)
For a flag:               vllm [subcommand] --help=max-model-len  (_ or - accepted)
Documentation:            https://docs.vllm.ai
23:45:17
5 Oct 2025
@daniel-fahey:matrix.orgDaniel FaheyI ended up rewriting the whole thing if you want to give it a spin and leave a review? https://github.com/NixOS/nixpkgs/pull/44882812:26:53
@daniel-fahey:matrix.orgDaniel Fahey

I've been using this one-liner while cobbling it together

git show cc6098112333e5ac645aa14f2ea9f70878d8fe22:pkgs/development/python-modules/vllm/default.nix \
> pkgs/development/python-modules/vllm/default.nix \
&& git diff pkgs/development/python-modules/vllm/default.nix \
&& ./pkgs/development/python-modules/vllm/update.sh \
&& git diff pkgs/development/python-modules/vllm/default.nix

cc6098112333e5ac645aa14f2ea9f70878d8fe22 being the Nixpkgs revision with vLLM v0.10.2, you can also test it with other revisions corresponding to other semantic versions. I almost went to town writing tests, but I'd have enough fun by then

12:29:44
@daniel-fahey:matrix.orgDaniel Fahey

yeah, see https://wiki.nixos.org/wiki/Nixpkgs/Update_Scripts

and you use e.g. passthru.updateScript = ./update.sh

(apparently), it's my first time writing one, will have to wait and see if @r-ryantm uses it

12:50:30
@daniel-fahey:matrix.orgDaniel Faheyone could also automate updating e.g. https://github.com/NixOS/nixpkgs/blob/107f8b572eb41058b610f99aba21b9a1b5925cf8/pkgs/development/python-modules/vllm/default.nix#L183-216, but I thought what I'd done was try-hard over-engineering enough already I really wanted to try and make a reference implementation that could easily be adapted to other complicated Python packages that have multiple git deps12:54:29
@daniel-fahey:matrix.orgDaniel Faheyhttps://www.explainxkcd.com/wiki/index.php/1319:_Automation12:55:42

Show newer messages


Back to Room ListRoom Version: 9