| 2 Nov 2025 |
 lillecarl | The only way to generate the nix-csi manifests currently is with easykubenix, it's poorly documented so when you're getting to please hit me up 😄 | 21:33:21 |
| lillecarl | The default.nix tries to build the CSI container image locally and ingest it into your containerd. You can ofc use easykubenix and just import the modules from $nix-csi/kubenix but I wouldn't expect most to grok that yet | 21:34:22 |
| lillecarl | * The default.nix tries to build the CSI container image locally and ingest it into your containerd. You can ofc use easykubenix and just import the modules from $nix-csi/kubenix but I wouldn't expect most to grok that yet
Edit: Borderline gatekeeping / ADHD last 20% issues :P | 21:34:56 |
 Erik | Definitely I will hit you up! | 21:40:37 |
| Erik | When it is the building done? | 21:41:16 |
| lillecarl | Easiest way actually is probably editing default.nix
nix-csi = {
namespace = "nix-csi";
image = imageRef; # comment this out
cache.storageClassName = "some nice CSI";
authorizedKeys = [
"YOUR KEY HERE :)"
];
ctest = {
enable = true;
replicas = 1;
};
};
Then running
nix run --file . kubenixEval.deploymentScript -- --prune --yes
| 21:41:39 |
| lillecarl | The CSI part is "done", lately I've been working on strapping some more things to it. The cache node sets up a loadbalancer and exposes itself over port 22, it has a populated /etc/nix/machines and with some SSH config on your client you can use the Kubernetes cluster as your own remote builder farm 😄 | 21:43:17 |
| lillecarl | There's more work to be done on the lifetime of paths in the cache, Nix SUCKS at garbage collection. There's only gcroots or go fuck yourself so I've patched it a bit so whenever the daemon does a path query it'll refresh "registraitonTime" in the nix database, then we use that to garbage collect instead of just killing everything that doesn't have a gcroot | 21:45:08 |
| lillecarl | It's a bit hacky atm, I'm looking to improve it significantly 😄 (Code quality wise) | 21:45:45 |
| lillecarl | * It's a bit hacky atm, I'm looking to improve it significantly 😄 (Code quality wise, and only doing the registrationTime update on certain Nix operations) | 21:46:15 |
| Erik | wow | 21:47:10 |
| Erik | I wasnt expecting to patch nix | 21:47:29 |
| Erik | hahahahaha | 21:47:30 |
| lillecarl | You don't have to, it's the one supplied in the nix-csi container 😄 | 21:48:03 |
| lillecarl | https://github.com/Lillecarl/lix/commit/b5e5a56b93da57239feb192416291af21df8dfe8 <- This is the patch, it's really quite simple: Whenever anything queries a path update registrationTime. And to clarify: You don't have to patch your own client, this is only relevant within the nodes and they speak the same nix-daemon protocol as anyone | 21:49:20 |
| lillecarl | And I apply the patch as an overlay on pkgs.lix so it's not stalled either. It will probably break in 2.94 when they're ripping ca-derivations out of Lix (but it's just that they're changing nearby code, nothing "actualy codey" that's relevant 😄 | 21:51:16 |
| Erik | No problem with that | 21:52:44 |
| lillecarl | * And I apply the patch as an overlay on pkgs.lix so it's not stalled either. It will probably break in 2.94 when they're ripping ca-derivations out of Lix (but it's just that they're changing nearby code, nothing "actualy codey" that's relevant 😄
Edit: You could ofc not patch Lix/Nix too, it's just that there's no good way to track when a path was last "hot" in the cache without something like this. I've talked to Rickard @nixbuild.net and they're doing the same thing but.... Less hacky 😄 | 21:52:49 |
| Erik | Great work | 21:53:12 |
