!VhbWwlUdjHkamKnfrK:nixos.org

Nix: Cloud Native

260 Members
65 Servers

Load older messages

SenderMessageTime
2 Nov 2025
@lillecarl:matrix.orglillecarlHaha yeah if Nix can't do it it can generate a script that does it 21:31:08
@lillecarl:matrix.orglillecarlThe only way to generate the nix-csi manifests currently is with easykubenix, it's poorly documented so when you're getting to please hit me up 😄21:33:21
@lillecarl:matrix.orglillecarlThe default.nix tries to build the CSI container image locally and ingest it into your containerd. You can ofc use easykubenix and just import the modules from $nix-csi/kubenix but I wouldn't expect most to grok that yet21:34:22
@lillecarl:matrix.orglillecarl* The default.nix tries to build the CSI container image locally and ingest it into your containerd. You can ofc use easykubenix and just import the modules from $nix-csi/kubenix but I wouldn't expect most to grok that yet Edit: Borderline gatekeeping / ADHD last 20% issues :P21:34:56
@erikeah:matrix.orgErikDefinitely I will hit you up! 21:40:37
@erikeah:matrix.orgErikWhen it is the building done?21:41:16
@lillecarl:matrix.orglillecarl

Easiest way actually is probably editing default.nix

            nix-csi = {
              namespace = "nix-csi";
              image = imageRef; # comment this out
              cache.storageClassName = "some nice CSI";
              authorizedKeys = [
                "YOUR KEY HERE :)"
              ];
              ctest = {
                enable = true;
                replicas = 1;
              };
            };

Then running

nix run --file . kubenixEval.deploymentScript -- --prune --yes
21:41:39
@lillecarl:matrix.orglillecarlThe CSI part is "done", lately I've been working on strapping some more things to it. The cache node sets up a loadbalancer and exposes itself over port 22, it has a populated /etc/nix/machines and with some SSH config on your client you can use the Kubernetes cluster as your own remote builder farm 😄21:43:17
@lillecarl:matrix.orglillecarlThere's more work to be done on the lifetime of paths in the cache, Nix SUCKS at garbage collection. There's only gcroots or go fuck yourself so I've patched it a bit so whenever the daemon does a path query it'll refresh "registraitonTime" in the nix database, then we use that to garbage collect instead of just killing everything that doesn't have a gcroot21:45:08
@lillecarl:matrix.orglillecarlIt's a bit hacky atm, I'm looking to improve it significantly 😄 (Code quality wise)21:45:45
@lillecarl:matrix.orglillecarl* It's a bit hacky atm, I'm looking to improve it significantly 😄 (Code quality wise, and only doing the registrationTime update on certain Nix operations)21:46:15
@erikeah:matrix.orgErikwow21:47:10
@erikeah:matrix.orgErikI wasnt expecting to patch nix21:47:29
@erikeah:matrix.orgErikhahahahaha21:47:30
@lillecarl:matrix.orglillecarlYou don't have to, it's the one supplied in the nix-csi container 😄21:48:03
@lillecarl:matrix.orglillecarlhttps://github.com/Lillecarl/lix/commit/b5e5a56b93da57239feb192416291af21df8dfe8 <- This is the patch, it's really quite simple: Whenever anything queries a path update registrationTime. And to clarify: You don't have to patch your own client, this is only relevant within the nodes and they speak the same nix-daemon protocol as anyone 21:49:20
@lillecarl:matrix.orglillecarlAnd I apply the patch as an overlay on pkgs.lix so it's not stalled either. It will probably break in 2.94 when they're ripping ca-derivations out of Lix (but it's just that they're changing nearby code, nothing "actualy codey" that's relevant 😄21:51:16
@erikeah:matrix.orgErikNo problem with that21:52:44
@lillecarl:matrix.orglillecarl* And I apply the patch as an overlay on pkgs.lix so it's not stalled either. It will probably break in 2.94 when they're ripping ca-derivations out of Lix (but it's just that they're changing nearby code, nothing "actualy codey" that's relevant 😄 Edit: You could ofc not patch Lix/Nix too, it's just that there's no good way to track when a path was last "hot" in the cache without something like this. I've talked to Rickard @nixbuild.net and they're doing the same thing but.... Less hacky 😄21:52:49
@erikeah:matrix.orgErikGreat work21:53:12
@lillecarl:matrix.orglillecarlThanks :)21:53:16
@lillecarl:matrix.orglillecarlI've put a lot of effort into it tbh 😄21:53:35
@erikeah:matrix.orgErikSince nix, i do not fear patches any more21:53:40
@lillecarl:matrix.orglillecarl *

Easiest way actually is probably editing default.nix

            nix-csi = {
              namespace = "nix-csi";
              image = imageRef; # comment this out
              cache.storageClassName = "some nice CSI";
              authorizedKeys = [
                "YOUR KEY HERE :)"
              ];
              ctest = {
                enable = false;
                replicas = 0;
              };
            };

Then running

nix run --file . kubenixEval.deploymentScript -- --prune --yes
21:54:55
@erikeah:matrix.orgErikI will give try and if get stuck i will get to you21:55:06
@lillecarl:matrix.orglillecarlI'm up for another 3-4 hours at least (CET)21:56:18
@erikeah:matrix.orgErikToday I will not be able, where I live is midnight22:00:44
@lillecarl:matrix.orglillecarl *

Easiest way actually is probably editing default.nix

            nix-csi = {
              namespace = "nix-csi";
              image = imageRef; # comment this out
              cache.storageClassName = "some nice CSI";
              authorizedKeys = [
                "YOUR KEY HERE :) (Only required if you wanna push to the cluster cache)"
              ];
              ctest = {
                enable = false;
                replicas = 0;
              };
            };

Then running

nix run --file . kubenixEval.deploymentScript -- --prune --yes
22:00:48
@lillecarl:matrix.orglillecarlNp! There's always another day, nuclear Armageddon is not on my agenda22:01:40
@erikeah:matrix.orgErikBuajajajajjaja22:02:07

Show newer messages

Back to Room ListRoom Version: 6