| 10 Feb 2025 |
snipped-paws | Hello, I am new-ish to NixOS and disko and am trying to get a setup working for my home server. I have this cut-down config that is just supposed to format the single mirror that I plan to install NixOS on top of, but it seems to not correctly create the zfs partition (as well as the zpool and datasets). When I run sudo disko --mode disko disko-config.nix, it appears to work (?), but when I check the partitions in Gnome Disks in the live environment, the content of the partition is listed as "Unknown." The EFI boot partition does appear to be recognized as expected. Any help would be much appreciated!
{
disko.devices = let
nixosPartitions = {
ESP = {
size = "512M";
type = "EF00";
content.type = "filesystem";
content.format = "vfat";
content.mountpoint = "/boot";
};
zfs = {
size = "100%";
content.type = "zfs";
content.pool = "nixos";
};
};
in {
disk = {
# NixOS boot mirror (240GBx2)
ssd0 = {
device = "/dev/sda";
type = "disk";
content = {
type = "gpt";
partitions = nixosPartitions;
};
};
ssd1 = {
device = "/dev/sdb";
type = "disk";
content = {
type = "gpt";
partitions = nixosPartitions;
};
};
};
};
zpool = {
# NixOS boot mirror
nixos = {
type = "zpool";
mode = "mirror";
options = { ashift = "12"; };
datasets = {
"root" = {
type = "zfs_fs";
mountpoint = "/";
options = {
encryption = "aes-256-gcm";
keyformat = "passphrase";
keylocation = "prompt";
};
};
"nix" = {
type = "zfs_fs";
mountpoint = "/nix";
options."com.sun:auto-snapshot" = "false";
};
"home" = {
type = "zfs_fs";
mountpoint = "/home";
options = {
encryption = "aes-256-gcm";
keyformat = "raw";
keylocation = "file:///etc/zfskey";
};
};
};
};
};
}
| 06:10:38 |
lassulus | Gnome disks can't show zfs partitions | 07:16:56 |
lassulus | Afaik | 07:17:01 |
snipped-paws | I've managed to make a little more progress:
{
disko.devices = let
nixosPartitions = {
ESP = {
size = "512M";
type = "EF00";
content.type = "filesystem";
content.format = "vfat";
content.mountpoint = "/boot";
};
zfs = {
size = "100%";
content.type = "zfs";
content.pool = "zpool-nixos";
};
};
in {
disk = {
# NixOS boot mirror (240GBx2)
ssd0 = {
device = "/dev/sda";
type = "disk";
content = {
type = "gpt";
partitions = nixosPartitions;
};
};
ssd1 = {
device = "/dev/sdb";
type = "disk";
content = {
type = "gpt";
partitions = nixosPartitions;
};
};
};
zpool = {
# NixOS boot mirror
zpool-nixos = {
type = "zpool";
mode = "mirror";
options = { ashift = "12"; };
preCreateHook = ''
# Create the temporary keyfile before any ZFS dataset is created
head -c 32 /dev/urandom > /run/tmp-zfskey
chmod 600 /run/tmp-zfskey
'';
datasets = {
"root" =
{ # NOTE: Don't encrypt root dataset directly to allow flexibility later.
type = "zfs_fs";
mountpoint = "/";
};
"root/unlock" = {
type = "zfs_fs";
mountpoint = "/unlock";
options = {
encryption = "aes-256-gcm";
keyformat = "passphrase";
keylocation = "prompt";
};
};
"root/nix" = {
type = "zfs_fs";
mountpoint = "/nix";
options = {
encryption = "aes-256-gcm";
keyformat = "raw";
keylocation = "file:///run/tmp-zfskey";
};
};
"root/home" = {
type = "zfs_fs";
mountpoint = "/home";
options = {
encryption = "aes-256-gcm";
keyformat = "raw";
keylocation = "file:///run/tmp-zfskey";
};
};
};
postCreateHook = ''
# Move the keyfile into the unlock dataset
# cp /run/tmp-zfskey /mnt/unlock/zfskey
# chmod 600 /mnt/unlock/zfskey
echo "postCreateHook running" >> /tmp/disko-debug.log
ls -lah /mnt/unlock >> /tmp/disko-debug.log 2>&1
echo "before" >> /tmp/disko-debug.log 2>&1
cp /run/tmp-zfskey /mnt/unlock/zfskey >> /tmp/disko-debug.log 2>&1
echo "after" >> /tmp/disko-debug.log 2>&1
chmod 600 /mnt/unlock/zfskey >> /tmp/disko-debug.log 2>&1
ls -lah /mnt/unlock >> /tmp/disko-debug.log 2>&1
# Update keylocation for other datasets (will be correct after reboot)
zfs set keylocation=file:///unlock/zfskey zpool-nixos/root/nix
zfs set keylocation=file:///unlock/zfskey zpool-nixos/root/home
'';
};
};
};
}
This mostly runs, but the postCreateHook has an issue with copying the key file. Not sure if I am going about this the right way, so any pointers would be much appreciated. I basically want to have just one dataset encrypted with a password, and then have all other encrypted datasets reference a key file stored on the password protected one so that I can unlock everything with just one password.
| 11:02:03 |
projectinitiative | Does anyone have an explanation on how the disk type and dependencies get tracked? Similar to how for mdadm and mdraid, mdraid runs before mdadm as that is a prereq. I am trying to do something similar but running into issues | 13:31:25 |
lassulus | there is deviceDependencies in _meta which gets exported and toposorted later | 13:36:10 |
lassulus | or later in that case when it's used | 13:36:26 |
projectinitiative | Hmm, I think I'm using that correctly. I am noticing one type's create method isn't executing at all. I added some debug statements to the shell script, and that's where I noticed my issue. Not at my computer atm, but I can send the example in a few | 13:54:18 |
projectinitiative | https://github.com/ProjectInitiative/disko/blob/feat/bcachefs-as-member/lib/types/bcachefs_member.nix#L57 | 14:33:37 |
| 11 Feb 2025 |
projectinitiative | Are there any good examples of how to use the topsorted function? Doing some digging in nix repl for the mdadm example, I see this behavior:
nix-repl> inputs.nixpkgs.lib.lists.last nixosConfigurations.testmachine.config.disko.devices._meta.dev[
"disk"
"/dev/my-disk"
]
nix-repl> builtins.head nixosConfigurations.testmachine.config.disko.devices._meta.deviceDependencies.mdadm.raid1
[
"disk"
"/dev/my-disk"
]
Seems a bit odd only one element is in the list. And I wonder if this is why my new types are not getting ordered properly, because they follow the same structure as mdadm/mdraid.
| 14:16:26 |
projectinitiative | * Are there any good examples of how to use the topsorted function? Doing some digging in nix repl for the mdadm example, I see this behavior:
nix-repl> inputs.nixpkgs.lib.lists.last nixosConfigurations.testmachine.config.disko.devices._meta.deviceDependencies.mdadm.raid1[
"disk"
"/dev/my-disk"
]
nix-repl> builtins.head nixosConfigurations.testmachine.config.disko.devices._meta.deviceDependencies.mdadm.raid1
[
"disk"
"/dev/my-disk"
]
Seems a bit odd only one element is in the list. And I wonder if this is why my new types are not getting ordered properly, because they follow the same structure as mdadm/mdraid.
| 14:16:59 |
projectinitiative | * Are there any good examples of how to use the topsorted function? Doing some digging in nix repl for the mdadm example, I see this behavior:
nix-repl> inputs.nixpkgs.lib.lists.last nixosConfigurations.testmachine.config.disko.devices._meta.deviceDependencies.mdadm.raid1
[
"disk"
"/dev/my-disk"
]
nix-repl> builtins.head nixosConfigurations.testmachine.config.disko.devices._meta.deviceDependencies.mdadm.raid1
[
"disk"
"/dev/my-disk"
]
Seems a bit odd only one element is in the list. And I wonder if this is why my new types are not getting ordered properly, because they follow the same structure as mdadm/mdraid.
| 14:17:08 |
projectinitiative | * Are there any good examples of how to use the sortDevicesByDependencies function? Doing some digging in nix repl for the mdadm example, I see this behavior:
nix-repl> inputs.nixpkgs.lib.lists.last nixosConfigurations.testmachine.config.disko.devices._meta.deviceDependencies.mdadm.raid1
[
"disk"
"/dev/my-disk"
]
nix-repl> builtins.head nixosConfigurations.testmachine.config.disko.devices._meta.deviceDependencies.mdadm.raid1
[
"disk"
"/dev/my-disk"
]
Seems a bit odd only one element is in the list. And I wonder if this is why my new types are not getting ordered properly, because they follow the same structure as mdadm/mdraid.
| 14:18:06 |
lassulus | it should just be the attribute you depend on, so in this case disko.devices.mdadm.raid1 depends on disko.devices.disk."/dev/my-disk" | 14:19:49 |
lassulus | and the toposort then sorts the snippet for disk."/dev/my-disk" before the snipper for mdadm.raid1 | 14:21:03 |
lassulus | but if your raid depends on multiple disks, there should be multiple disks in there | 14:23:18 |
projectinitiative | looking at example/mdadm.nix, two disks are defined with mdadm and then mdraid as the content, shouldn't both be included?
I guess ultimately I am trying to figure out why my types that are modeled after the zfs and mdadm patterns are not respecting the same order when they get combined in the final _create. i.e. the mdraid equivalent bcachefs_member type (for members of the pool) needs to come first, yet it gets put after the bachefs type I made. Both types are nearly identical in structure | 14:23:25 |
projectinitiative | I will note I also tried with the zfs-with-vdevs example adding it to the test machine config, and it also only has one disk listed under dependencies despite having several listed in the example file under zroot | 14:24:54 |
lassulus | hmm, maybe there is indeed a bug | 14:25:56 |
lassulus | which didn't get triggered yet, but I thought the complex example would take care of that | 14:26:17 |
projectinitiative | reproduction steps:
nixosConfigurations.testmachine = lib.nixosSystem {
system = "x86_64-linux";
modules = [
./tests/disko-install/configuration.nix
# ./example/hybrid.nix
./module.nix
# ./example/bcachefs-multi-disk.nix
# ./example/mdadm.nix
./example/zfs-with-vdevs.nix
];
- add different examples to load and play around with
- load the flake into nix repl
- look at some of the _meta.deviceDependencies
| 14:27:29 |
projectinitiative | * reproduction steps:
nixosConfigurations.testmachine = lib.nixosSystem {
system = "x86_64-linux";
modules = [
./tests/disko-install/configuration.nix
# ./example/hybrid.nix
./module.nix
# ./example/bcachefs-multi-disk.nix
# ./example/mdadm.nix
./example/zfs-with-vdevs.nix
];
- add different examples to load and play around with toggling various configs
- load the flake into nix repl
- look at some of the _meta.deviceDependenciese
| 14:27:59 |
lassulus | I will try to add that to my thaigersprint todo list tomorrow :D | 14:28:16 |
projectinitiative | Obligatory: I wish nix had better testing/debugging tools, this is taking way longer to nail down my overall issue haha... | 14:30:10 |
shift | In reply to @lassulus:lassul.us I will try to add that to my thaigersprint todo list tomorrow :D You want issues for your sprint? Haha | 20:09:11 |
| 12 Feb 2025 |
lassulus | You have more? :D | 01:38:19 |
projectinitiative | In reply to @lassulus:lassul.us I will try to add that to my thaigersprint todo list tomorrow :D I looked up that thaigersprint was, that seems unique. Talk about a cool experience | 03:48:41 |
projectinitiative | * I looked up what thaigersprint was, that seems unique. Talk about a cool experience | 03:48:49 |
lassulus | ok, deepMergeMap actually doesn't merge lists, since recursiveUpdate doesn't take care of that | 05:31:21 |
lassulus | projectinitiative: https://github.com/nix-community/disko/pull/963 | 10:21:50 |