!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

286 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda57 Servers

Load older messages


SenderMessageTime
4 Oct 2025
@daniel-fahey:matrix.orgDaniel FaheyNo one likes us, we don't care! We are Nix, super Nix https://repology.org/project/python%3Avllm/versions14:14:54
@daniel-fahey:matrix.orgDaniel Fahey

Gaétan Lepage: see https://github.com/NixOS/nixpkgs/compare/master...daniel-fahey:nixpkgs:update-vllm

What's best, I'll supersede your PR?

19:18:57
@glepage:matrix.orgGaétan LepageI merged the vllm bump PR.19:19:22
@glepage:matrix.orgGaétan Lepage Now I'm working on bumping it to v0.11.0 19:19:33
@daniel-fahey:matrix.orgDaniel FaheyMe too lol19:20:28
@glepage:matrix.orgGaétan LepageWill open the PR in a minute19:20:59
@daniel-fahey:matrix.orgDaniel FaheyCool, I'm writing up what I've discovered19:25:37
@longregen:matrix.orglonRedacted or Malformed Event22:10:50
@daniel-fahey:matrix.orgDaniel Fahey

lon: oh you deleted, (hehe), I actually didn't even think about it and had completely forgotten nixpkgs didn't have CUDA 13 yet (https://github.com/NixOS/nixpkgs/pull/437723)

Thanks for looking into it all the same

22:29:03
@glepage:matrix.orgGaétan Lepage Daniel Fahey FYI vllm is broken on master as my PR was merged slightly too soon. 23:12:03
@glepage:matrix.orgGaétan LepageIf you have a bit of time to investigate, please go on :)23:12:12
@longregen:matrix.orglonYes, sorry I deleted because I saw your commit and is the same as mine (save for the update script! I didn't know that was a pattern people in nixpkgs used, TIL)23:13:31
@longregen:matrix.orglonthe nvidia/cutlass dependency can also be updated fwiw, with the update script 23:18:40
@longregen:matrix.orglonimage.png
Download image.png
23:18:43
@daniel-fahey:matrix.orgDaniel Faheyyeah, just started rewriting it23:23:25
@daniel-fahey:matrix.orgDaniel FaheyHow can you tell? Hydra? Got a link?23:27:58
@daniel-fahey:matrix.orgDaniel Fahey

Looks okay for me, some other problem? CUDA build?

[daniel@laptop:~/Source/nixpkgs]$ nix-build -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/b967613ed760449a73eaa73d7b69eb45e857ce1a.tar.gz --expr 'with import <nixpkgs> { }; python313Packages.vllm'
unpacking 'https://github.com/NixOS/nixpkgs/archive/b967613ed760449a73eaa73d7b69eb45e857ce1a.tar.gz' into the Git cache...
/nix/store/amncczb34wd5zingwclr3sqa6q7kahay-python3.13-vllm-0.11.0

[daniel@laptop:~/Source/nixpkgs]$ ./result/bin/vllm --help
INFO 10-05 00:44:00 [__init__.py:216] Automatically detected platform cpu.
usage: vllm [-h] [-v] {chat,complete,serve,bench,collect-env,run-batch} ...

vLLM CLI

positional arguments:
  {chat,complete,serve,bench,collect-env,run-batch}
    chat                Generate chat completions via the running API server.
    complete            Generate text completions based on the given prompt via the running API server.
    collect-env         Start collecting environment information.
    run-batch           Run batch prompts and write results to file.

options:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit

For full list:            vllm [subcommand] --help=all
For a section:            vllm [subcommand] --help=ModelConfig    (case-insensitive)
For a flag:               vllm [subcommand] --help=max-model-len  (_ or - accepted)
Documentation:            https://docs.vllm.ai
23:45:17
5 Oct 2025
@daniel-fahey:matrix.orgDaniel FaheyI ended up rewriting the whole thing if you want to give it a spin and leave a review? https://github.com/NixOS/nixpkgs/pull/44882812:26:53
@daniel-fahey:matrix.orgDaniel Fahey

I've been using this one-liner while cobbling it together

git show cc6098112333e5ac645aa14f2ea9f70878d8fe22:pkgs/development/python-modules/vllm/default.nix \
> pkgs/development/python-modules/vllm/default.nix \
&& git diff pkgs/development/python-modules/vllm/default.nix \
&& ./pkgs/development/python-modules/vllm/update.sh \
&& git diff pkgs/development/python-modules/vllm/default.nix

cc6098112333e5ac645aa14f2ea9f70878d8fe22 being the Nixpkgs revision with vLLM v0.10.2, you can also test it with other revisions corresponding to other semantic versions. I almost went to town writing tests, but I'd have enough fun by then

12:29:44
@daniel-fahey:matrix.orgDaniel Fahey

yeah, see https://wiki.nixos.org/wiki/Nixpkgs/Update_Scripts

and you use e.g. passthru.updateScript = ./update.sh

(apparently), it's my first time writing one, will have to wait and see if @r-ryantm uses it

12:50:30
@daniel-fahey:matrix.orgDaniel Faheyone could also automate updating e.g. https://github.com/NixOS/nixpkgs/blob/107f8b572eb41058b610f99aba21b9a1b5925cf8/pkgs/development/python-modules/vllm/default.nix#L183-216, but I thought what I'd done was try-hard over-engineering enough already I really wanted to try and make a reference implementation that could easily be adapted to other complicated Python packages that have multiple git deps12:54:29
@daniel-fahey:matrix.orgDaniel Faheyhttps://www.explainxkcd.com/wiki/index.php/1319:_Automation12:55:42
@daniel-fahey:matrix.orgDaniel FaheyvLLM is becoming a huge project and pillar in the ecosystem, sometimes their cadence for releases is daily, and going through and checking each is fiddly and tedious. May well save us all some time, but it's good to just not have to worry https://www.explainxkcd.com/wiki/index.php/1205:_Is_It_Worth_the_Time%3F hope it keeps working, lol13:01:36
@daniel-fahey:matrix.orgDaniel Fahey * vLLM is becoming a huge project and pillar in the ecosystem, sometimes their cadence for releases is daily, and going through and checking each dep is fiddly and tedious. May well save us all some time, but it's good to just not have to worry https://www.explainxkcd.com/wiki/index.php/1205:_Is_It_Worth_the_Time%3F hope it keeps working, lol 13:02:02
@daniel-fahey:matrix.orgDaniel FaheyCUDA build! Great to see it on your Hydra server though. I might have time in the evenings this week to get to the bottom of it, but don't have access to some beefy compute with 128+ GB RAM and 16+ CPU to do the CUDA build in a reasonable timeframe https://hydra.nixos-cuda.org/build/824/nixlog/7113:29:19
@daniel-fahey:matrix.orgDaniel Fahey* CUDA build! Great to see it on your Hydra server though. I might have time in the evenings this week to get to the bottom of it, but don't have access to some beefy compute with 128+ GB RAM and 16+ CPU to do the CUDA build in a reasonable timeframe (going to be investing in an Epyc rackmount rig eventually, as my Nixbuild.net bill can get silly fast) https://hydra.nixos-cuda.org/build/824/nixlog/7113:31:47
@daniel-fahey:matrix.orgDaniel FaheyHad a quick look while on the Tube, failure cascade stems from FlashMLA build15:47:38
@daniel-fahey:matrix.orgDaniel Fahey I think vLLM v0.11.0 CUDA build fails because FlashMLA's SM100 (Blackwell) code requires CUTLASS v4.2.1+ APIs like make_counting_tensor, but the derivation uses CUTLASS v4.0.0.
So I just disable the problematic SM100 kernels, builds chugging along, seems to have got further than the one on Hydra, going to get a PR up now.
21:45:40
@daniel-fahey:matrix.orgDaniel Fahey * I think vLLM v0.11.0 CUDA build fails because FlashMLA's SM100 (Blackwell) code requires CUTLASS v4.2.1+ APIs like make_counting_tensor, but the derivation uses CUTLASS v4.0.0.
So I just disabled the problematic SM100 kernels, build's chugging along, seems to have got further than the one on Hydra, going to get a PR up now.
21:46:01
@glepage:matrix.orgGaétan Lepage I didn't bump cutlass because vllm 0.11.0 still uses v4.0.0:
https://github.com/vllm-project/vllm/blob/v0.11.0/CMakeLists.txt#L273
21:53:29

Show newer messages


Back to Room ListRoom Version: 9