| 18 Jul 2024 |
|  @ironbound:hackerspace.pl changed their display name from Professor Bin Dong to Professor. | 22:49:10 |
| @ironbound:hackerspace.pl changed their display name from Professor to P_Big. | 22:49:51 |
| 19 Jul 2024 |
|  shapr joined the room. | 19:25:24 |
|  @fishball02:matrix.org left the room. | 21:30:09 |
| 21 Jul 2024 |
 hexa (UTC+1) | can anyone here test a hardening change for ollama with rocm? | 18:33:12 |
| hexa (UTC+1) | https://github.com/NixOS/nixpkgs/pull/328713 | 18:33:17 |
| hexa (UTC+1) | they want at least a vega 56, but not the rx 5000 series | 18:33:44 |
| hexa (UTC+1) | services.ollama = {
enable = true;
acceleration = "rocm";
};
| 18:34:17 |
| hexa (UTC+1) | then run ollama pull gemma | 18:35:26 |
| hexa (UTC+1) | and | 18:36:00 |
| hexa (UTC+1) | curl http://localhost:11434/api/generate -d '{
"model": "gemma",
"prompt":"Why is the sky blue?"
}'
| 18:36:09 |
| hexa (UTC+1) | * services.ollama = {
enable = true;
acceleration = "rocm";
};
systemd.services.ollama.serviceConfig = {
DeviecPolicy = "closed";
DeviceAllow = [
"/dev/dri/renderD128"
"/dev/dri/renderD129"
"/dev/dri/renderD130"
"/dev/dri/renderD131"
/dev/kfd"
};
};
| 18:39:42 |
| hexa (UTC+1) | * services.ollama = {
enable = true;
acceleration = "rocm";
};
systemd.services.ollama.serviceConfig = {
DeviecPolicy = "closed";
DeviceAllow = [
"/dev/dri/renderD128"
"/dev/dri/renderD129"
"/dev/dri/renderD130"
"/dev/dri/renderD131"
"/dev/kfd"
};
};
| 18:39:49 |
|  mindstorms6 joined the room. | 18:43:32 |
|  adamcstephens joined the room. | 19:05:53 |
| hexa (UTC+1) | SomeoneSerge (UTC+3): you know, I wonder if we can just keep these device node lists for cuda and rocm in a central location | 20:58:59 |
| adamcstephens | In reply to @hexa:lossy.network
can anyone here test a hardening change for ollama with rocm? the serviceConfig changes in that work for me | 23:54:28 |
| adamcstephens | In reply to @hexa:lossy.network
can anyone here test a hardening change for ollama with rocm? * the serviceConfig changes in your PR work for me | 23:55:12 |
| hexa (UTC+1) | adamcstephens 🐝: with what hardware/acceleration? | 23:55:46 |
| adamcstephens | rocm on a 6700 XT | 23:56:07 |
| hexa (UTC+1) | because for me (rx5700) and atemu (rx6700) it fails to find a device | 23:56:18 |
| hexa (UTC+1) | level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 rocm]"
level=WARN source=amd_linux.go:58 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
level=WARN source=amd_linux.go:186 msg="amdgpu too old gfx000" gpu=0
level=INFO source=amd_linux.go:345 msg="no compatible amdgpu devices detected"
| 23:56:41 |
| hexa (UTC+1) | it is related to DeviceAllow=/DevicePolicy | 23:56:57 |
| hexa (UTC+1) | * it is related to DeviceAllow=/DevicePolicy= | 23:56:58 |
 SomeoneSerge (utc+3) | In reply to @hexa:lossy.network
SomeoneSerge (UTC+3): you know, I wonder if we can just keep these device node lists for cuda and rocm in a central location Sounds like "why aren't we doing this yet?" | 23:57:05 |
| hexa (UTC+1) | yes, why aren't we? 😄 | 23:57:25 |
| hexa (UTC+1) | you see that a lot of work goes into discovery of these | 23:57:33 |
| adamcstephens | Jul 21 19:43:20 sink1 ollama[3567]: time=2024-07-21T19:43:20.566-04:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 rocm]"
Jul 21 19:43:20 sink1 ollama[3567]: time=2024-07-21T19:43:20.566-04:00 level=INFO source=gpu.go:205 msg="looking for compatible GPUs"
Jul 21 19:43:20 sink1 ollama[3567]: time=2024-07-21T19:43:20.567-04:00 level=WARN source=amd_linux.go:58 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
Jul 21 19:43:20 sink1 ollama[3567]: time=2024-07-21T19:43:20.568-04:00 level=INFO source=amd_linux.go:333 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
Jul 21 19:43:20 sink1 ollama[3567]: time=2024-07-21T19:43:20.568-04:00 level=INFO source=types.go:103 msg="inference compute" id=0 library=rocm compute=gfx1031 driver=0.0 name=1002:73df total="12.0 GiB" available="9.8 GiB"
| 23:59:37 |
| 22 Jul 2024 |
| hexa (UTC+1) | sus | 00:01:56 |
| hexa (UTC+1) | can you post deviceallow/policy? | 00:02:04 |
