!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

324 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda64 Servers

Load older messages


SenderMessageTime
11 Feb 2023
@ss:someonex.netSomeoneSerge (matrix works sometimes)
In reply to @FRidh:matrix.org
The curse of purity
Are you familiar with Guix's "grafts"? I just overheard about them, they seem to aim at solving a similar issue
13:27:13
@jovulic:matrix.orgjvYeah, I think it would have just worked if I was using docker or podman. Really, podman was pretty close, it is just that containerd is using cgroupsv2 and the podman config.toml has no-cgroups set to true.13:30:13
@jovulic:matrix.orgjvBut alas. I am working with the kubernetes config in nixpkgs, which is using containerd.13:30:53
@FRidh:matrix.orgFRidhYes, I think this was based on https://github.com/NixOS/rfcs/pull/315:33:45
@FRidh:matrix.orgFRidhor some earlier proposal from nbp15:34:38
@connorbaker:matrix.orgconnor (he/him) Would anyone have the bandwidth to review https://github.com/NixOS/nixpkgs/pull/215552 and https://github.com/NixOS/nixpkgs/pull/215549 (which depends on the previous one)?
It doesn't look to me like the @NixOS/cuda-maintainers tag is working unfortunately and I wasn't sure who to tag.
19:18:02
@winterqt:nixos.devWinter (she/her) connor (he/him): Looks like the ping worked. Would also help if you indicated that you tested it, as the template asks you to do... 19:19:14
13 Feb 2023
@ss:someonex.netSomeoneSerge (matrix works sometimes)
In reply to @FRidh:matrix.org
The curse of purity

The price of a particular kind of composability, more like it šŸ¤”

If we gave up "being able to deploy multiple versions of the same library" in favour of invalidating the conflicting combinations of libraries, we could have a "pure" system for quickly generating arbitrary FHS trees. We could also rely on $ORIGIN more than on absolute paths. Could even be a better trade-off for all this tensorflow non-sense

15:46:08
15 Feb 2023
@connorbaker:matrix.orgconnor (he/him) It looks like CUDNN 8.8.0 is available as redistributables (https://developer.download.nvidia.com/compute/redist/cudnn/v8.8.0/local_installers/11.8/) but they're no longer providing -archive.tar.xz files. Because I'm trying to update nixpkgs with the new version, should I look into unpacking an RPM or DEB file? 21:07:38
16 Feb 2023
@FRidh:matrix.orgFRidh
In reply to @connorbaker:matrix.org
It looks like CUDNN 8.8.0 is available as redistributables (https://developer.download.nvidia.com/compute/redist/cudnn/v8.8.0/local_installers/11.8/) but they're no longer providing -archive.tar.xz files. Because I'm trying to update nixpkgs with the new version, should I look into unpacking an RPM or DEB file?
posted this at the conda-forge feedstock repo. They typically have some contact with NVIDIA https://github.com/conda-forge/cudnn-feedstock/issues/54#issuecomment-1432630957
07:18:10
17 Feb 2023
@justbrowsing:matrix.orgKevin Mittman (UTC-7)
In reply to @connorbaker:matrix.org
It looks like CUDNN 8.8.0 is available as redistributables (https://developer.download.nvidia.com/compute/redist/cudnn/v8.8.0/local_installers/11.8/) but they're no longer providing -archive.tar.xz files. Because I'm trying to update nixpkgs with the new version, should I look into unpacking an RPM or DEB file?
ughh sounds like a bug
16:46:55
@justbrowsing:matrix.orgKevin Mittman (UTC-7) https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/linux-x86_64/ I see the tarballs here ... unless misunderstandingĀ  16:49:47
@justbrowsing:matrix.orgKevin Mittman (UTC-7) the cuda11 one is meant for all of cuda 11.x 16:51:05
@connorbaker:matrix.orgconnor (he/him)Did they... move the downloads?16:52:45
@justbrowsing:matrix.orgKevin Mittman (UTC-7)

Yes, this was done for consistency. /compute/<product>/redist/<component>/Ā 

Ā 

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/#tarball-and-zip-archive-deliverables

17:03:43
20 Feb 2023
@connorbaker:matrix.orgconnor (he/him)

How do you all handle PRs which have dependencies on other PRs?

For context, this is related to a PR I made here (https://github.com/NixOS/nixpkgs/pull/215229) which has spiraled into a grab-bag of changes.

As an example, if you have a PR A and a PR B which relies on that changes A has, how do you make that clear?

  • Should B use the branch A creates instead of Nixpkgs' master?
  • Should there be notes or warnings in the description of the PRs about the merge order? If so, is there a particular style you'd recommend?
  • How do you test multiple PRs which depend on each other with nixpkgs-review? I had tried specifying multiple PRs as arguments, but it didn't seem like it stacked them.

The closest thing I think I've seen before is PRs against torch (example: https://github.com/pytorch/pytorch/pull/94243). They use this tool: https://github.com/ezyang/ghstack.

15:14:57
@connorbaker:matrix.orgconnor (he/him) Additionally, when there are no maintainers and a package is failing (because of an unrelated issue) on master, who would you recommend tagging for review? For example, caffe: https://github.com/NixOS/nixpkgs/pull/217330 17:21:18
@me:linj.techlinj
In reply to @connorbaker:matrix.org
Additionally, when there are no maintainers and a package is failing (because of an unrelated issue) on master, who would you recommend tagging for review? For example, caffe: https://github.com/NixOS/nixpkgs/pull/217330
I would tag the people who have contributed to the file in my pr for review. If my pr gets approvels, I would tag currently active committers (people who just merged some prs) for merging.
17:38:27
@FRidh:matrix.orgFRidh
In reply to @connorbaker:matrix.org

How do you all handle PRs which have dependencies on other PRs?

For context, this is related to a PR I made here (https://github.com/NixOS/nixpkgs/pull/215229) which has spiraled into a grab-bag of changes.

As an example, if you have a PR A and a PR B which relies on that changes A has, how do you make that clear?

  • Should B use the branch A creates instead of Nixpkgs' master?
  • Should there be notes or warnings in the description of the PRs about the merge order? If so, is there a particular style you'd recommend?
  • How do you test multiple PRs which depend on each other with nixpkgs-review? I had tried specifying multiple PRs as arguments, but it didn't seem like it stacked them.

The closest thing I think I've seen before is PRs against torch (example: https://github.com/pytorch/pytorch/pull/94243). They use this tool: https://github.com/ezyang/ghstack.

That's always tough. I think your approach here, all in one PR but with clear commits is fine. The thing is, there are just not that many contributors in Nixpkgs to CUDA and it's quite a big PR so it takes some effort to review.
18:47:27
@connorbaker:matrix.orgconnor (he/him)https://github.com/NixOS/nixpkgs/pull/217322 is closer to what I envision doing once I've further split apart that large PR. Does that seem okay? Is there an automated tool I should be using to do something similar to this?19:03:27
@connorbaker:matrix.orgconnor (he/him) Thank you all for the work you do maintaining the CUDA-accelerated packages. Building jaxlib and tensorflowWithCuda repeatedly is awful. 21:45:11
21 Feb 2023
@hexa:lossy.networkhexahttps://github.com/NixOS/nixpkgs/pull/21749716:55:31
@connorbaker:matrix.orgconnor (he/him)Has anyone used or set up CCACHE for any of the CUDA derivations? I know they take a while to build and I'm curious what's been done to try to reduce build times21:07:03
@ss:someonex.netSomeoneSerge (matrix works sometimes)Hi! Thank you for investing your time and work in this right now!21:20:45
@ss:someonex.netSomeoneSerge (matrix works sometimes) Interesting. I just looked up the ccache nixos wiki page, it suggests one can just drop in something called ccacheStdenv 21:21:46
@ss:someonex.netSomeoneSerge (matrix works sometimes) Do you know if packages built that way would work as substitutes for normal stdenv ones? 21:22:59
@connorbaker:matrix.orgconnor (he/him)Unfortunately I think it would be a different derivation :l21:35:02
@ss:someonex.netSomeoneSerge (matrix works sometimes)so, no magic(21:57:53
@connorbaker:matrix.orgconnor (he/him)From a conversation I had, it seems like it's intended more for use as a dev-shell than for the end derivation23:00:04
22 Feb 2023
@hexa:lossy.networkhexaso, I have an application that wants tflite_runtime00:17:18

Show newer messages


Back to Room ListRoom Version: 9