I AI'd if --impure can be enabled by default and just now stumbled on a feature I hadn't heard about before! The option allow-unsafe-native-code-during-evaluation gives you a new builtin function called builtins.exec which can be used like this:

nix-repl> :p builtins.fromJSON (builtins.exec [ ./test.fish ])
{
  active = true;
  age = 25;
  name = "test-user";
  tags = [
    "developer"
    "nix"
    "json"
  ];
}

# test.fish
#! /usr/bin/env fish

echo "''"
echo '
{
  "name": "test-user",
  "age": 25,
  "active": true,
  "tags": ["developer", "nix", "json"]
}'
echo "''"

(It parses output as a Nix expression so i had to wrap the json in '' '' so it's treated as a string by Nix)

I'm definitely gonna dive deeper into usecases for this, as everything Nix it'll stall evaluation so don't do extremely slow things :)

In reply to @lillecarl:matrix.org
For home/personal use --impure should be the default IMO. You can use builtins.getEnv to read environment variables and you can read files anywhere on the filesystem, not just the sacred repo.
(Non-flake users don't have this problem since everything is --impure by default :))

In reply to @claesatwork:matrix.org
thanks, I will try to show you next time!

In reply to @lillecarl:matrix.org
Nice! Remembered https://search.nixos.org/options?channel=unstable&query=zramSwap ? :)

In reply to @claesatwork:matrix.org
I will definitely enable it on at least one of my NixOS hosts. Seems too good to be true :-)

Virtual memory is dark magic only special kernel wizards with superpowers can comprehend. All I know is those wizards want you to have a good amount of swap space so they can perform their spells without constraints.

(Honestly actually really: Swap is essential even if you have boatloads of RAM since it uses LRU to keep hot shit in fast memory and yesteryears Zara collection on the bench, everyone who says you don't need swap haven't spoken to the superpowered wizards)

In reply to @lillecarl:matrix.org
https://asciinema.org/a/I6RXVGmCyCcHnXInvh5DKph4C
read-write nix store in kubernetes with overlayfs 😄 yayy

In reply to @claesatwork:matrix.org
It would be interesting to see the data you pipe into kubectl apply in the first command, it seems to be the magic sauce :-)

https://gist.github.com/Lillecarl/cbd2e037f1fba7c145a37c3f33d5f926

Essentially everything except the daemonset,csidriver and the ubuntu pod is reduntant(in the current state). Currently the volume is just "ephemeral" so the expression metadata is straight in the podspec under volumeAttributes rather than through the expression CRD.

The reason for having a CRD to stick nix expressions into is so that they can be built ahead of pod-creation time, but I've reprioritized to just make sure it works reliably building on pod creation. It'll still be "kewl enough" to hopefully attract more eyes and ideas 😄

Claes: The entire CSI driver is in this file: https://github.com/Lillecarl/cknix/blob/main/cknix_csi/cknix.py

I've cleaned it up now and honestly it's so dumb and simple now. The "coolest" part is probably the execline script (Don't wanna ship a full shell) that pipes nix-store --dump-db to nix-store --load-db 😸
https://github.com/Lillecarl/cknix/blob/main/flake.nix#L63

The nix code quality is still garbage so don't judge too hard

In reply to @claesatwork:matrix.org
Cool!

In the example you exec into the pod to show that the nix store is present. Do you have a plan for how it will work for a pod to execute the right process at start?

I assume all pods running in a cluster will all have the same nix store mounted so the loose end here is to distinguish which pods run what?

Nix makes the "top derivation" available under /nix/var/result/bin/binaryname in the container, which can be set as the command in podspec to execute straight from the volume. Volumes are mounted before the container is created so it works out (You should use "tini" as the entrypoint which then executes your code to reap zombie processes, same thing with any container really).

How the stores work: There's a daemonset (run once per node) that is this CSI driver, it mounts HOST/var/lib/cknix/nix to /nix2 in the daemonset initcontainer, copies /nix to /nix2. Then the actual CSI container runs and mounts HOST/var/lib/cknix/nix to /nix. This store is is the "source store" for all pods on that NODE. But to create an isolated "store view" per pod so they can't see other pods storepaths we create "fake stores" using hardlinks only containing the paths from nix path-info into the "fake store" the "fake store" is then either bind mounted (shares page cache and is amazing but readonly) into the pod, or overlayfs mounted (means you can run nix commands within the workload pod and RW the entire Nix store :)

RW mounting should only be in development, but it's cool because it allows you to edit any storepaths (since the workload pods aren't privileged we can't setup a nix-daemon and do the RO bind mount thing on /nix/store so the entire store is read-write so you can do shenanigans you've never before imagined without strangling your dog