| 8 May 2025 |
 Kevin Mittman (EOY sleep) | In reply to @ss:someonex.net
RE: Why
How do we know if we can assume that (pname, version, platform) always corresponds to a unique hash, e.g. that nvidia never publishes two different things under the same (versioned) name? (treating sbsa and jetson as different platforms)
Well so far I haven't found any exceptions: A v. B
There is a policy of version uniqueness. Unfortunately, enforcement is limited | 18:26:54 |
| Kevin Mittman (EOY sleep) | In reply to @connorbaker:matrix.org
Fair question; I don’t think we do.
Is there a concern you had in mind?
For what it’s worth, I’ve seen them use the “source” and “linux-all” platform (meta-platform?) to indicate a package works on multiple platforms rather than having an entry for each (all with the same hash). In the rework I give those preferential treatment since they’re agnostic and don’t seem to occur alongside other variants of the package: https://github.com/ConnorBaker/cuda-packages/blob/7731ebdf397c5ce123571abf2dc41ebac86237d3/pkgs/development/cuda-modules/packages/redist-builder/package.nix#L102 driver_assistant is a pure Python script, hence linux-all. and cudnn_samples is source code, that would ideally be posted on Github | 18:31:25 |
 SomeoneSerge (back on matrix) | Well I want functionality for "merging" a set of manifests, rather than having single cuda manifest first spawn a package set, which is then extended with other components like cudnn. I think that suggests a merged domain of "package( recipe)s" which the future package set can grab stuff from. Finally, if things are to share a scope, how do we know they don't compete for the same spot? ^^^ | 18:35:02 |
| SomeoneSerge (back on matrix) | Yeah as long as we ensure observability on our side I'm fine making this an assumption | 18:37:23 |
 connor (burnt/out) (UTC-8) | Up to you! | 19:30:27 |
| connor (burnt/out) (UTC-8) | Merging in what sense?
As an example, in cuda-packages I traverse the collection of manifests provided for to an instance of a CUDA package set, creating a arguments to be fed to redist-builder: https://github.com/ConnorBaker/cuda-packages/blob/62a4c5b58eed0038d73ac2b5e6bc151d851c0f81/pkgs/development/cuda-modules/default.nix#L73-L91 | 19:35:23 |
| Kevin Mittman (EOY sleep) | I can imagine it would be useful to have a list of all cuda12 compatible CUDA-X binary archives | 22:24:06 |
| SomeoneSerge (back on matrix) | Merging as in merging manifests into a single manifest, single hierarchy, as opposed to constructing a set of packages from a sequence of overlays | 22:51:35 |
| SomeoneSerge (back on matrix) | In particular, I'd prefer there be a single set of "archive"s, all of the same shape, and each have a unique key. E.g. it could be a tree where archives.${pname}.${platform}.${version}.${compatTag} (the tags are for cudnn...) is { relative_path: String, sha256: String, md5: Option<String> }, etc. Note no ${manifest_name} in there. | 22:57:56 |
| SomeoneSerge (back on matrix) | So we found a trivial violation:) | 22:58:50 |
| SomeoneSerge (back on matrix) | There's also no reason we should assume there's ever at most one compatibility tag... | 22:59:39 |
| 9 May 2025 |
| Kevin Mittman (EOY sleep) | could you provide a pseudo-code form of what you are asking for? | 00:50:30 |
| Kevin Mittman (EOY sleep) | specifically trying to ask if there would be one or multiple versions of each component to satisfy the dependency closure? | 00:58:31 |
 Guillaume Desforges | Hey! Has anyone tried to package torchcodec?
https://github.com/pytorch/torchcodec | 17:07:26 |
| Guillaume Desforges | * Hey! Has anyone tried to package torchcodec?
https://github.com/pytorch/torchcodec
(sorry if that's the wrong channel) | 17:22:33 |
| Guillaume Desforges | * Hey! Has anyone tried to package torchcodec?
https://github.com/pytorch/torchcodec
(sorry if that's the wrong channel, afaik it's ok since it's NVIDIA-related) | 17:22:48 |
| Guillaume Desforges | * Hey! Has anyone tried to package torchcodec?
https://github.com/pytorch/torchcodec
(sorry if that's the wrong channel, afaik it's ok since it's torch/NVIDIA-related) | 17:23:13 |
| Guillaume Desforges | * Hey! Has anyone tried to package torchcodec?
https://github.com/pytorch/torchcodec
(sorry if that's the wrong channel, afaik it's ok since it's torch/CUDA-related) | 17:23:25 |
| connor (burnt/out) (UTC-8) | :L
on master:
nix-repl> legacyPackages.x86_64-linux.cudaPackages.cudnn.src
«derivation /nix/store/7gwjjblxcjxbbj8ry5zir6m35gks3aq6-cudnn-linux-aarch64-9.8.0.87_cuda12-archive.tar.xz.drv»
| 18:01:59 |
| connor (burnt/out) (UTC-8) | lmao somehow https://github.com/NixOS/nixpkgs/commit/9fd753ea84e5035b357a275324e7fd7ccfb1fc77 caused this | 18:09:44 |
| connor (burnt/out) (UTC-8) | I'll work on a fix shortly | 18:17:22 |
| connor (burnt/out) (UTC-8) | hmm, in hindsight the removed and changed packages listed here were a red flag: https://github.com/NixOS/nixpkgs/actions/runs/14912284892/attempts/1#summary-41889732827 | 18:18:44 |
| connor (burnt/out) (UTC-8) | Okay! https://github.com/NixOS/nixpkgs/pull/405707 should fix it --it also cleans up a fair amount of the logic. | 20:23:06 |
| connor (burnt/out) (UTC-8) | Also, if someone would mind reviewing https://github.com/NixOS/nixpkgs/pull/405664 | 21:02:13 |
| connor (burnt/out) (UTC-8) | Gaétan Lepage: any chance you'd be able to look at this? I'd like to get it merged soon since things depending on cuDNN (like PyTorch) are broken on x86_64-linux. | 21:22:03 |
 Gaétan Lepage | I started to look at it but got scared by the nature of the diff | 21:22:53 |
| Gaétan Lepage | Will look at it :) | 21:22:58 |
| connor (burnt/out) (UTC-8) | <3 | 21:23:01 |
| connor (burnt/out) (UTC-8) | If it helps at all, the evaluatedModules.config.${pname}.releases.${redistArch} attribute set we're indexing into is essentially just the releases.nix file each of the multiplexed packages provides (for example, here's cuDNN's: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/cuda-modules/cudnn/releases.nix). | 21:26:01 |
| connor (burnt/out) (UTC-8) | The call to lib.foldl' inside newestPackages traverses the list of package (the elements of the list of attribute sets underneath each architecture in releases.nix), keeping the latest copy of each major-minor version. | 21:28:59 |
