| 7 May 2025 |
 SomeoneSerge (back on matrix) | But sometimes they feel the need for dramaclean separation of binaries for different platforms and they hurt each other | 23:31:50 |
 connor (burnt/out) (UTC-8) | CMake? Sure... mostly. rapids-cmake (https://github.com/rapidsai/rapids-cmake)? Absolutely not :( | 23:35:37 |
 Tristan Ross | CMake isn't my friend | 23:38:24 |
| connor (burnt/out) (UTC-8) | Alrighty, so I'm changing overrides back to fixups (and making a note about how there's no "fixup" for versioned packages since they only apply to the base package, by pname) and floating the imports in pkgs/top-level/cuda-packages.nix back down and inline | 23:38:42 |
| connor (burnt/out) (UTC-8) | Does that sound right SomeoneSerge (UTC+U[-12,12])? | 23:38:51 |
| SomeoneSerge (back on matrix) | That's what I had in mind too | 23:44:34 |
| 8 May 2025 |
| connor (burnt/out) (UTC-8) | Alrighty, should be good once CI passes | 00:04:29 |
| connor (burnt/out) (UTC-8) | PyTorch still built, so that's good | 00:04:37 |
| SomeoneSerge (back on matrix) | RE: Why
How do we know if we can assume that (pname, version, platform) always corresponds to a unique hash, e.g. that nvidia never publishes two different things under the same (versioned) name? (treating sbsa and jetson as different platforms)
Well so far I haven't found any exceptions: A v. B
| 02:32:49 |
| SomeoneSerge (back on matrix) | (didn't load all manifests yet tho) | 02:33:21 |
| connor (burnt/out) (UTC-8) | Fair question; I don’t think we do.
Is there a concern you had in mind?
For what it’s worth, I’ve seen them use the “source” and “linux-all” platform (meta-platform?) to indicate a package works on multiple platforms rather than having an entry for each (all with the same hash). In the rework I give those preferential treatment since they’re agnostic and don’t seem to occur alongside other variants of the package: https://github.com/ConnorBaker/cuda-packages/blob/7731ebdf397c5ce123571abf2dc41ebac86237d3/pkgs/development/cuda-modules/packages/redist-builder/package.nix#L102 | 04:54:54 |
| connor (burnt/out) (UTC-8) | Okay bed time for me.
Tomorrow will include work on introducing the setup hook changes | 04:59:20 |
| connor (burnt/out) (UTC-8) | Thank you for the fixes Serge! | 16:07:04 |
| SomeoneSerge (back on matrix) | Sure, thank you! | 17:10:55 |
| SomeoneSerge (back on matrix) | I avoided force-pushes. Do you think we should squash? | 17:11:27 |
 Kevin Mittman (UTC-8) | In reply to @ss:someonex.net
RE: Why
How do we know if we can assume that (pname, version, platform) always corresponds to a unique hash, e.g. that nvidia never publishes two different things under the same (versioned) name? (treating sbsa and jetson as different platforms)
Well so far I haven't found any exceptions: A v. B
There is a policy of version uniqueness. Unfortunately, enforcement is limited | 18:26:54 |
| Kevin Mittman (UTC-8) | In reply to @connorbaker:matrix.org
Fair question; I don’t think we do.
Is there a concern you had in mind?
For what it’s worth, I’ve seen them use the “source” and “linux-all” platform (meta-platform?) to indicate a package works on multiple platforms rather than having an entry for each (all with the same hash). In the rework I give those preferential treatment since they’re agnostic and don’t seem to occur alongside other variants of the package: https://github.com/ConnorBaker/cuda-packages/blob/7731ebdf397c5ce123571abf2dc41ebac86237d3/pkgs/development/cuda-modules/packages/redist-builder/package.nix#L102 driver_assistant is a pure Python script, hence linux-all. and cudnn_samples is source code, that would ideally be posted on Github | 18:31:25 |
| SomeoneSerge (back on matrix) | Well I want functionality for "merging" a set of manifests, rather than having single cuda manifest first spawn a package set, which is then extended with other components like cudnn. I think that suggests a merged domain of "package( recipe)s" which the future package set can grab stuff from. Finally, if things are to share a scope, how do we know they don't compete for the same spot? ^^^ | 18:35:02 |
| SomeoneSerge (back on matrix) | Yeah as long as we ensure observability on our side I'm fine making this an assumption | 18:37:23 |
| connor (burnt/out) (UTC-8) | Up to you! | 19:30:27 |
| connor (burnt/out) (UTC-8) | Merging in what sense?
As an example, in cuda-packages I traverse the collection of manifests provided for to an instance of a CUDA package set, creating a arguments to be fed to redist-builder: https://github.com/ConnorBaker/cuda-packages/blob/62a4c5b58eed0038d73ac2b5e6bc151d851c0f81/pkgs/development/cuda-modules/default.nix#L73-L91 | 19:35:23 |
| Kevin Mittman (UTC-8) | I can imagine it would be useful to have a list of all cuda12 compatible CUDA-X binary archives | 22:24:06 |
| SomeoneSerge (back on matrix) | Merging as in merging manifests into a single manifest, single hierarchy, as opposed to constructing a set of packages from a sequence of overlays | 22:51:35 |
| SomeoneSerge (back on matrix) | In particular, I'd prefer there be a single set of "archive"s, all of the same shape, and each have a unique key. E.g. it could be a tree where archives.${pname}.${platform}.${version}.${compatTag} (the tags are for cudnn...) is { relative_path: String, sha256: String, md5: Option<String> }, etc. Note no ${manifest_name} in there. | 22:57:56 |
| SomeoneSerge (back on matrix) | So we found a trivial violation:) | 22:58:50 |
| SomeoneSerge (back on matrix) | There's also no reason we should assume there's ever at most one compatibility tag... | 22:59:39 |
| 9 May 2025 |
| Kevin Mittman (UTC-8) | could you provide a pseudo-code form of what you are asking for? | 00:50:30 |
| Kevin Mittman (UTC-8) | specifically trying to ask if there would be one or multiple versions of each component to satisfy the dependency closure? | 00:58:31 |
 Guillaume Desforges | Hey! Has anyone tried to package torchcodec?
https://github.com/pytorch/torchcodec | 17:07:26 |
| Guillaume Desforges | * Hey! Has anyone tried to package torchcodec?
https://github.com/pytorch/torchcodec
(sorry if that's the wrong channel) | 17:22:33 |
