| 19 Jan 2025 |
 connor (burnt/out) (UTC-8) | Interesting byproduct of getting rid of the CUDA stdenv is having libraries produced by multiple versions of a compiler, which breaks LTO: https://github.com/ConnorBaker/cuda-packages/commit/87cb387ecfba3f46b76e604b896d64417fdf9e8b | 16:15:00 |
 Gaétan Lepage | In reply to @hexa:lossy.network
https://hydra.nix-community.org/project/nixpkgs So on the nix-community CI builders ? | 17:01:35 |
 hexa | yes | 17:03:53 |
| Gaétan Lepage | Ok thanks | 17:05:39 |
 SomeoneSerge (back on matrix) | https://github.com/NixOS/nixpkgs/pull/369956#issuecomment-2601068826
Maybe .override { cudaSupport = true; } should have been .override { stdenv = cudaStdenv; } in the first place. All lib.optionals cudaSupport should have been lib.optionals stdenv.isCuda or stdenv.isNVCC. We could remove (move to stdenv) all cuda- or hip-related flags, but the easiest way to do that is to make config.cudaSupport replace the actual default stdenv for all packages, triggering even more rebuilds than we do now.
Accidentally, this would also make it easier to move config.cudaSupport into crossSystem and introduce a pkgsCross.cuda.
| 23:36:30 |
| 21 Jan 2025 |
|  kaya 𖤐 changed their profile picture. | 00:27:42 |
|  Joel joined the room. | 12:12:46 |
 caniko | Does ollama still require compiling CUDA? | 20:16:41 |
| caniko | I failed to compile last time, and actually have no interest in doing so... | 20:17:44 |
| SomeoneSerge (back on matrix) | It's optional | 21:13:16 |
|  oak 🏳️🌈♥️ changed their profile picture. | 22:35:37 |
| SomeoneSerge (back on matrix) | * CUDA support is optional. If you enable it you have to rebuild ollama. It should be cached in nix-community cachix | 21:14:50 |
| oak 🏳️🌈♥️ changed their profile picture. | 22:37:17 |
| SomeoneSerge (back on matrix) | * CUDA support is optional. If you enable it you have to rebuild ollama. It should be cached in nix-community cachix (more specifically, (import <nixpkgs> { config.cudaSupport = true; }).ollama should be) | 21:15:17 |
| 23 Jan 2025 |
| SomeoneSerge (back on matrix) | connor (he/him) (UTC-7): we never published that community update, you think you'd be able to find the time to push that over the line? | 14:37:45 |
| 24 Jan 2025 |
| connor (burnt/out) (UTC-8) | Ah yeah; do you remember if I had a draft or if you did? | 06:31:29 |
| SomeoneSerge (back on matrix) | In reply to @ss:someonex.net
https://pad.lassul.us/DwBgo1SZSzSVL29wsnK0fg# ^ | 08:52:02 |
|  @leighleighleigh:matrix.org left the room. | 20:40:08 |
| 25 Jan 2025 |
| caniko | In reply to @ss:someonex.net
CUDA support is optional. If you enable it you have to rebuild ollama. It should be cached in nix-community cachix (more specifically, (import <nixpkgs> { config.cudaSupport = true; }).ollama should be) Really? It used compile stuff in C back when I did this the last time | 14:31:28 |
| caniko | perhaps something changed? | 14:31:42 |
| caniko | I just checked, it has parts that are written in C 😄 | 14:33:59 |
| caniko | does nix get the build requirements by itself? | 14:34:35 |
| SomeoneSerge (back on matrix) | The build recipe is described in nixpkgs, and its behaviour can be modified by a few parameters: https://github.com/NixOS/nixpkgs/blob/7b0f4d83a902d8718df0f09bad5c34670b72cda6/pkgs/by-name/ol/ollama/package.nix#L32-L40 | 14:37:53 |
| SomeoneSerge (back on matrix) | (import <nixpkgs> { config.cudaSupport = true; }).ollama in nixos-unstable seems to build fine, idk about the runtime: https://hydra.nix-community.org/build/2851634 | 14:39:00 |
| caniko | OK, very cool. Thank you | 14:41:56 |
| caniko | will you work towards caching ollama with cuda? | 14:42:12 |
| SomeoneSerge (back on matrix) | The build results of this hydra instance are cached in https://nix-community.org/cache/ | 14:42:43 |
| caniko | wow, it is cached already 😄 | 14:43:27 |
| caniko | OK, great. | 14:43:36 |
| SomeoneSerge (back on matrix) | Yes, thanks to the community-funded nix-community project: https://discourse.nixos.org/t/cuda-cache-for-nix-community/56038 | 15:01:25 |
