!VhbWwlUdjHkamKnfrK:nixos.org

Nix: Cloud Native

248 Members
56 Servers

Load older messages


SenderMessageTime
2 Nov 2025
@lillecarl:matrix.orglillecarlhttps://github.com/Lillecarl/lix/commit/b5e5a56b93da57239feb192416291af21df8dfe8 <- This is the patch, it's really quite simple: Whenever anything queries a path update registrationTime. And to clarify: You don't have to patch your own client, this is only relevant within the nodes and they speak the same nix-daemon protocol as anyone 21:49:20
@lillecarl:matrix.orglillecarlAnd I apply the patch as an overlay on pkgs.lix so it's not stalled either. It will probably break in 2.94 when they're ripping ca-derivations out of Lix (but it's just that they're changing nearby code, nothing "actualy codey" that's relevant 😄21:51:16
@erikeah:matrix.orgErikNo problem with that21:52:44
@lillecarl:matrix.orglillecarl* And I apply the patch as an overlay on pkgs.lix so it's not stalled either. It will probably break in 2.94 when they're ripping ca-derivations out of Lix (but it's just that they're changing nearby code, nothing "actualy codey" that's relevant 😄 Edit: You could ofc not patch Lix/Nix too, it's just that there's no good way to track when a path was last "hot" in the cache without something like this. I've talked to Rickard @nixbuild.net and they're doing the same thing but.... Less hacky 😄21:52:49
@erikeah:matrix.orgErikGreat work21:53:12
@lillecarl:matrix.orglillecarlThanks :)21:53:16
@lillecarl:matrix.orglillecarlI've put a lot of effort into it tbh 😄21:53:35
@erikeah:matrix.orgErikSince nix, i do not fear patches any more21:53:40
@lillecarl:matrix.orglillecarl *

Easiest way actually is probably editing default.nix

            nix-csi = {
              namespace = "nix-csi";
              image = imageRef; # comment this out
              cache.storageClassName = "some nice CSI";
              authorizedKeys = [
                "YOUR KEY HERE :)"
              ];
              ctest = {
                enable = false;
                replicas = 0;
              };
            };

Then running

nix run --file . kubenixEval.deploymentScript -- --prune --yes
21:54:55
@erikeah:matrix.orgErikI will give try and if get stuck i will get to you21:55:06
@lillecarl:matrix.orglillecarlI'm up for another 3-4 hours at least (CET)21:56:18
@erikeah:matrix.orgErikToday I will not be able, where I live is midnight22:00:44
@lillecarl:matrix.orglillecarl *

Easiest way actually is probably editing default.nix

            nix-csi = {
              namespace = "nix-csi";
              image = imageRef; # comment this out
              cache.storageClassName = "some nice CSI";
              authorizedKeys = [
                "YOUR KEY HERE :) (Only required if you wanna push to the cluster cache)"
              ];
              ctest = {
                enable = false;
                replicas = 0;
              };
            };

Then running

nix run --file . kubenixEval.deploymentScript -- --prune --yes
22:00:48
@lillecarl:matrix.orglillecarlNp! There's always another day, nuclear Armageddon is not on my agenda22:01:40
@erikeah:matrix.orgErikBuajajajajjaja22:02:07
@erikeah:matrix.orgErik @lillecarl:matrix.org: thanks for the hints! 22:04:24
3 Nov 2025
@jennydaman:fedora.imJennings Zhang left the room.18:53:49
4 Nov 2025
@cafkafk:gitter.imcafkafk changed their profile picture.08:23:35
7 Nov 2025
@mahmoudk1000:matrix.orgMahmoud joined the room.19:06:56
@emma:rory.gayEmma [it/its] left the room.22:41:28
9 Nov 2025
@lillecarl:matrix.orglillecarl

Erik: I reworked where patched Lix updates paths and made it update recursively so now it sucks less ™️

https://github.com/Lillecarl/lix/commit/9ac72bbd0c7802ca83a907d1fec135f31aab6d24

I'm gonna add a script on a timer to the CSI pods that runs nix path-info --all | nix copy --to nix-cache. This way the cache will always hold any storePaths alive in the cluster while not requiring any complex bookkeeping (nix copy acts as a cache refresher) 😄

It's shaping up pretty darn nice, especially the feature of using the CSI pods as a build farm

22:02:01
10 Nov 2025
@lillecarl:matrix.orglillecarlI'm building my own Hetzner Kubernetes (ClusterAPI managed) cluster using some of the Nix Kubernetes tools I've been working on (nix-csi and by proxy dinix, easykubenix). The goal of the cluster is to be the cheapest shit you could possibly imagine meaning no LoadBalancers. To do this I had to write a little Python thing that creates ipaddresspools for the MetalLB(controller no speaker) to assign service IP's from the nodes externalip list. I expanded it to split the IPv6 /64 you get from Hetzner into two and assign one to services and one to pods. Anyways to deploy this Python script I used nix-csi so here's a public example of how it works https://github.com/Lillecarl/hetzkube/commit/f35b8fadfe17d5cc7054ae6f6cf187879272d2ba Since the cluster is x86 and arm I had to build the script for both x86 and arm which was no problem (nix-csi picks arch by where it's scheduled) 😄 Aye it also makes DNSEndpoint resources from the control-plane nodes IP's so I don't need an LB there either02:58:48
@lillecarl:matrix.orglillecarl* I'm building my own Hetzner Kubernetes (ClusterAPI managed) cluster using some of the Nix Kubernetes tools I've been working on (nix-csi and by proxy dinix, easykubenix). The goal of the cluster is to be the cheapest shit you could possibly imagine meaning no LoadBalancers. To do this I had to write a little Python thing that creates ipaddresspools for the MetalLB(controller no speaker) to assign service IP's from the nodes externalip list. I expanded it to split the IPv6 /64 you get from Hetzner into two and assign one to services and one to pods. Anyways to deploy this Python script I used nix-csi so here's a public example of how it works https://github.com/Lillecarl/hetzkube/commit/f35b8fadfe17d5cc7054ae6f6cf187879272d2ba Since the cluster is x86 and arm I had to build the script for both x86 and arm which was no problem (nix-csi picks arch by where it's scheduled) 😄 Aye it also makes DNSEndpoint resources from the control-plane nodes IP's so I don't need an LB there either Disclaimer: The Python script is a collaboration with AI02:59:28
@lillecarl:matrix.orglillecarl * I'm building my own Hetzner Kubernetes (ClusterAPI managed) cluster using some of the Nix Kubernetes tools I've been working on (nix-csi and by proxy dinix, easykubenix). The goal of the cluster is to be the cheapest shit you could possibly imagine meaning no LoadBalancers. To do this I had to write a little Python thing that creates ipaddresspools for the MetalLB(controller no speaker) to assign service IP's from the nodes externalip list. I expanded it to split the IPv6 /64 you get from Hetzner into two and assign one to services and one to pods. Anyways to deploy this Python script I used nix-csi so here's a public example of how it works
commit Since the cluster is x86 and arm I had to build the script for both x86 and arm which was no problem (nix-csi picks arch by where it's scheduled) 😄) Here's the [nix-csi magic](https://github.com/Lillecarl/hetzkube/commit/f35b8fadfe17d5cc7054ae6f6cf187879272d2ba#diff-4c88f6039362165e9d0e418fce1c757befb04cf659d7738a89807fc7504ccadcR5-R113
Aye it also makes DNSEndpoint resources from the control-plane nodes IP's so I don't need an LB there either
Disclaimer: The Python script is a collaboration with AI
03:00:13
@lillecarl:matrix.orglillecarl * I'm building my own Hetzner Kubernetes (ClusterAPI managed) cluster using some of the Nix Kubernetes tools I've been working on (nix-csi and by proxy dinix, easykubenix). The goal of the cluster is to be the cheapest shit you could possibly imagine meaning no LoadBalancers. To do this I had to write a little Python thing that creates ipaddresspools for the MetalLB(controller no speaker) to assign service IP's from the nodes externalip list. I expanded it to split the IPv6 /64 you get from Hetzner into two and assign one to services and one to pods. Anyways to deploy this Python script I used nix-csi so here's a public example of how it works
commit Since the cluster is x86 and arm I had to build the script for both x86 and arm which was no problem (nix-csi picks arch by where it's scheduled) 😄) Here's the [nix-csi magic](https://github.com/Lillecarl/hetzkube/commit/f35b8fadfe17d5cc7054ae6f6cf187879272d2ba#diff-4c88f6039362165e9d0e418fce1c757befb04cf659d7738a89807fc7504ccadcR5-R113)
Aye it also makes DNSEndpoint resources from the control-plane nodes IP's so I don't need an LB there either
Disclaimer: The Python script is a collaboration with AI
03:00:27
@lillecarl:matrix.orglillecarl * I'm building my own Hetzner Kubernetes (ClusterAPI managed) cluster using some of the Nix Kubernetes tools I've been working on (nix-csi and by proxy dinix, easykubenix). The goal of the cluster is to be the cheapest shit you could possibly imagine meaning no LoadBalancers. To do this I had to write a little Python thing that creates ipaddresspools for the MetalLB(controller no speaker) to assign service IP's from the nodes externalip list. I expanded it to split the IPv6 /64 you get from Hetzner into two and assign one to services and one to pods. Anyways to deploy this Python script I used nix-csi so here's a public example of how it works
commit Since the cluster is x86 and arm I had to build the script for both x86 and arm which was no problem (nix-csi picks arch by where it's scheduled) 😄) Here's the nix-csi magic
Aye it also makes DNSEndpoint resources from the control-plane nodes IP's so I don't need an LB there either
Disclaimer: The Python script is a collaboration with AI
03:00:49
@lillecarl:matrix.orglillecarlI felt it was time to build an actual "production-ish" cluster to eventually host something real. Ofc it should run both DualStack IP and dual architectures to make it a little challenging03:03:36
@arianvp:matrix.orgArianDoes Hetzner do BGP any cast?18:11:00
11 Nov 2025
@robert:scs.ems.hostRobert Rose changed their display name from Robert Rose - 🏝️ bis 10.11.25 to Robert Rose.09:22:17
@xentec:gated.spacexentec | metal1nk@39c3 joined the room.14:05:12

Show newer messages


Back to Room ListRoom Version: 6