| 21 Jan 2025 |
|  kaya 𖤐 changed their profile picture. | 00:27:42 |
|  Joel joined the room. | 12:12:46 |
 caniko | Does ollama still require compiling CUDA? | 20:16:41 |
| caniko | I failed to compile last time, and actually have no interest in doing so... | 20:17:44 |
 SomeoneSerge (back on matrix) | It's optional | 21:13:16 |
|  oak 🏳️🌈♥️ changed their profile picture. | 22:35:37 |
| SomeoneSerge (back on matrix) | * CUDA support is optional. If you enable it you have to rebuild ollama. It should be cached in nix-community cachix | 21:14:50 |
| oak 🏳️🌈♥️ changed their profile picture. | 22:37:17 |
| SomeoneSerge (back on matrix) | * CUDA support is optional. If you enable it you have to rebuild ollama. It should be cached in nix-community cachix (more specifically, (import <nixpkgs> { config.cudaSupport = true; }).ollama should be) | 21:15:17 |
| 23 Jan 2025 |
| SomeoneSerge (back on matrix) | connor (he/him) (UTC-7): we never published that community update, you think you'd be able to find the time to push that over the line? | 14:37:45 |
| 24 Jan 2025 |
 connor (he/him) | Ah yeah; do you remember if I had a draft or if you did? | 06:31:29 |
| SomeoneSerge (back on matrix) | In reply to @ss:someonex.net
https://pad.lassul.us/DwBgo1SZSzSVL29wsnK0fg# ^ | 08:52:02 |
|  @leighleighleigh:matrix.org left the room. | 20:40:08 |
| 25 Jan 2025 |
| caniko | In reply to @ss:someonex.net
CUDA support is optional. If you enable it you have to rebuild ollama. It should be cached in nix-community cachix (more specifically, (import <nixpkgs> { config.cudaSupport = true; }).ollama should be) Really? It used compile stuff in C back when I did this the last time | 14:31:28 |
| caniko | perhaps something changed? | 14:31:42 |
| caniko | I just checked, it has parts that are written in C 😄 | 14:33:59 |
| caniko | does nix get the build requirements by itself? | 14:34:35 |
| SomeoneSerge (back on matrix) | The build recipe is described in nixpkgs, and its behaviour can be modified by a few parameters: https://github.com/NixOS/nixpkgs/blob/7b0f4d83a902d8718df0f09bad5c34670b72cda6/pkgs/by-name/ol/ollama/package.nix#L32-L40 | 14:37:53 |
| SomeoneSerge (back on matrix) | (import <nixpkgs> { config.cudaSupport = true; }).ollama in nixos-unstable seems to build fine, idk about the runtime: https://hydra.nix-community.org/build/2851634 | 14:39:00 |
| caniko | OK, very cool. Thank you | 14:41:56 |
| caniko | will you work towards caching ollama with cuda? | 14:42:12 |
| SomeoneSerge (back on matrix) | The build results of this hydra instance are cached in https://nix-community.org/cache/ | 14:42:43 |
| caniko | wow, it is cached already 😄 | 14:43:27 |
| caniko | OK, great. | 14:43:36 |
| SomeoneSerge (back on matrix) | Yes, thanks to the community-funded nix-community project: https://discourse.nixos.org/t/cuda-cache-for-nix-community/56038 | 15:01:25 |
| connor (he/him) | Good news: I got some experience writing tests for setup hooks in https://github.com/NixOS/nixpkgs/pull/370750, so I’m writing tests this weekend for the CUDA setup hooks I have in https://github.com/ConnorBaker/cuda-packages | 18:58:37 |
| connor (he/him) | Kevin Mittman: I don’t know if you’re able to raise this internally, but https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#deprecated-toolchains seems like it should have a list of toolchains or something instead of being empty. Are you able to, or would you mind, seeing if it’s supposed to be empty? | 19:03:47 |
| 26 Jan 2025 |
 hexa | wandb build failure on python-updates https://hydra.lossy.network/build/2649332 | 22:26:22 |
| hexa | not sure anyone here cares about that, but samuela used to | 22:26:52 |
| 27 Jan 2025 |
 Kevin Mittman (UTC-8) | In reply to @connorbaker:matrix.org
Kevin Mittman: I don’t know if you’re able to raise this internally, but https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#deprecated-toolchains seems like it should have a list of toolchains or something instead of being empty. Are you able to, or would you mind, seeing if it’s supposed to be empty? Fixed (for some definition of that) | 06:08:56 |
