| 12 Nov 2025 |
 Gaétan Lepage | Flox is managing their cache internally. As you pointed, they use an internal fork of nixpkgs that is slightly delayed from nixos-unstable (or nixos-unstable-small). It's normal that their cache is less fresh than chache.nixos-cuda.org. | 16:13:41 |
| Gaétan Lepage | The difference is that they have the permission from Nvidia to redistribute their binaries. | 16:14:01 |
 Daniel Fahey | 😅 | 16:14:45 |
 Ari Lotter | i have it in both because i don't understand it <3 | 16:15:52 |
| Ari Lotter | also - 100% sure i'm not running out of ram anymore, at only ~400gb/2tb used on the machine, but jax still has crashed workers - and not sure if it's progressing | 16:17:00 |
| Ari Lotter | can i pull up interactive logs for its derivation somehow? | 16:17:11 |
| Ari Lotter | oh nice, i got a real failure | 16:17:50 |
| Daniel Fahey | 
Download image.png | 16:18:16 |
| Daniel Fahey | Another option Ari, is you follow the "Help" button next to the closure link at https://hydra.nixos-cuda.org/build/8123
| 16:18:17 |
| Daniel Fahey | Could just pipe output through tee, but normally at the tail of the the crash, it has the command to run to see the log | 16:19:19 |
| Daniel Fahey | I'd be interested to see them, see if they're similar to what I saw on nixbuild.net -> you said it crashed during the test stage? | 16:19:53 |
| Daniel Fahey | *phase | 16:19:56 |
| Daniel Fahey | I also wonder if nixpkgs-review is complicating things | 16:20:33 |
| Ari Lotter | grabbing logs! | 16:22:05 |
| Ari Lotter | Download test.txt | 16:27:01 |
| Ari Lotter | that's a big log lol | 16:27:04 |
| Ari Lotter | but yeah, i hit both some INTERNAL: Failed to materialize symbols and some LLVM compilation error: Cannot allocate memory 🤔 | 16:28:19 |
| Ari Lotter | (keeping in mind i had >1600gb of ram available at all times) | 16:28:52 |
| Daniel Fahey | Might be different than what I saw on nixbuild.net but not sure, did your new attempt crashed building the python3.13 version, so not 3.12 like yesterday
The (server grade) Intel hypothesis still isn't disproven, tbf | 16:40:31 |
| Ari Lotter | 3.12 version is stuck on building python3.12-jax-0.8.0 (pytestCheckPhase): replacing crashed worker gw1 | 16:41:02 |
| Daniel Fahey | Have you fed it into a clanker yet? | 16:41:04 |
| Ari Lotter | still runnin | 16:41:05 |
| Ari Lotter | god i hate that term T_T | 16:41:22 |
| Daniel Fahey | ah, full crash is imminent | 16:41:30 |
| Ari Lotter | claude is down :p | 16:41:55 |
| Daniel Fahey | lol, I'm an Iain M. Banks Culture stan | 16:41:58 |
| Ari Lotter | i'm of the belief that being mean to anything, even inanimate objects like a rock or an LLM or a piece of software someone else wrote, is bad for your "soul" x3 | 16:42:42 |
| Ari Lotter | anyways | 16:42:44 |
| Daniel Fahey | You're absolutely right! | 16:43:17 |
| Daniel Fahey | I still have a hunch it's a problem when using newer server grade Intel chips for the build | 16:44:24 |
