| 4 Feb 2022 |
Buckley | yeah, provisioning seems inherently incompatible with statelessness | 20:47:19 |
CRTified | Buckley: I have found a decent solution, I think | 20:47:56 |
CRTified | eval-config has an additional parameter extraModules https://github.com/NixOS/nixpkgs/blob/a0ad8dcd354c67f084511e4ae78a27af83df95fd/nixos/lib/eval-config.nix#L28 - That allows you to add your usual modules to modules, and colmena.nixosModules.deploymentOptions to extraModules | 20:48:45 |
CRTified | So in general it looks like this for me now:
[---SNIP---]
sys = system: mods:
nixpkgs.lib.nixosSystem {
inherit system;
specialArgs = {inherit inputs; } ;
extraModules = [
colmena.nixosModules.deploymentOptions
];
modules = [
self.nixosModule
home-manager.nixosModules.home-manager
./common
] ++ mods;
};
in rec {
colmena = {
meta = {
description = "My personal machines";
nixpkgs = import nixpkgs {
system = "x86_64-linux";
overlays = [ nur.overlay ];
};
};
} // builtins.mapAttrs (name: value: {
nixpkgs.system = value.config.nixpkgs.system;
imports = value._module.args.modules;
}) (self.nixosConfigurations);
[---SNAP---]
| 20:49:46 |
Buckley | i think my ideal scenario would be for nixosConfigurations to import the colmena module explicitly, and then nxiosConfigurations to be read directly by colmena... though i do not claim this to be an easy lift :D | 20:51:41 |
Buckley | ive poked around the codebase a bit in hopes of writing some PRs but i just haven't put enough time into the rust stuff to figure it out yet | 20:52:41 |
CRTified | In reply to @buckley310:matrix.org i think my ideal scenario would be for nixosConfigurations to import the colmena module explicitly, and then nxiosConfigurations to be read directly by colmena... though i do not claim this to be an easy lift :D Yes, that would be the "best case", but I think I'm happy with my shim-method now 🙂 | 20:56:18 |
Buckley | shims for days 😎 | 21:00:59 |
@github:maunium.net | [zhaofengli/colmena] cole-h opened
issue
#54: Don't require meta.nixpkgs if meta.nodeNixpkgs is set for the target host(s)
I think it's a little weird to require nixpkgs to be set (because it requires a system set which may change at any point, depending on where you're deploying from). Sure, right now, I may be using x86_64-linux, but I may want to use aarch64-linux or some other arch at any time.
I would expect, if all of the target hosts have a matching nodeNixpkgs entry, that meta.nixpkgs wouldn't be required.
Even better, if it was combined with a deployment.nixpkgs option that replaces the meta.nodeNixpkgs option.
I may be able to work on this, if this is desired by more than just me.
| 23:48:40 |
@github:maunium.net | [zhaofengli/colmena] cole-h opened
issue
#55: Move meta.nodeNixpkgs to deployment.nixpkgs inside the node config
IMHO, it makes more sense to configure the used nixpkgs inside of the node's config, instead of in the meta attrset.
| 23:56:28 |
@github:maunium.net | [zhaofengli/colmena] cole-h edited
issue
#54: Don't require meta.nixpkgs if meta.nodeNixpkgs is set for the target host(s)
| 23:56:56 |
| 5 Feb 2022 |
@github:maunium.net | [zhaofengli/colmena] Repo forked into cole-h/colmena | 01:06:39 |
@github:maunium.net | [zhaofengli/colmena] cole-h drafted
pull request
#56: WIP: add deployment.nixpkgs option
See https://github.com/zhaofengli/colmena/issues/55.
| 01:08:13 |
@github:maunium.net | [zhaofengli/colmena] cole-h commented on
pull request
#56: WIP: add deployment.nixpkgs option | 01:10:06 |
@github:maunium.net | [zhaofengli/colmena] cole-h edited
pull request
#56: WIP: add deployment.nixpkgs option
| 01:16:27 |
@github:maunium.net | [zhaofengli/colmena] zhaofengli commented on
pull request
#56: WIP: add deployment.nixpkgs option : Please use the following as a test:
let
nixpkgs_2111 = builtins.fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/f6ddd55d5f9d5eca08df138c248008c1ba73ecec.tar.gz";
sha256 = "03ms1mk4m4ni3bxjzdqngxvwyvh3rn7yb8ybzjlg62r9iv3l0ppa";
};
nixpkgs_unstable = builtins.fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/554d2d8aa25b6e583575459c297ec23750adb6cb.tar.gz";
sha256 = "01yfqslnkyrfb5yjfablhvw830iw0za3mab4n03a0ldyq5ac6wh1";
};
in {
meta.nixpkgs = import nixpkgs_2111 { system = "x86_64-linux"; };
alpha = { pkgs, ... }: {
deployment.nixpkgs = import nixpkgs_unstable {
system = "x86_64-linux";
};
# Does not exist in 21.11
nix.settings.trusted-users = [ "zhaofeng" ];
boot.isContainer = true;
};
}
| 02:07:08 |
Zhaofeng Li | Wow, that was noisy 🙉 Let me turn off the comment and edit notifications... | 03:14:01 |
Zhaofeng Li | Well, actually I can't just disable the webhook event, since the "Pull requests" event encompasses PR opening (which I want to appear), (un)drafting as well as editing of the description. It has to be done from the bot's end which isn't very configurable 🤷♂️ | 03:19:16 |
| 9 Feb 2022 |
Winter (she/her) | Zhaofeng Li: How did you determine that the change in e58dde1be08cf7fbac7299e928b6d0e143edbec9 would fix #50? Like, what was your debug process? | 16:15:32 |
Winter (she/her) | curious | 16:15:34 |
Zhaofeng Li | In reply to @winterqt:nixos.dev Zhaofeng Li: How did you determine that the change in e58dde1be08cf7fbac7299e928b6d0e143edbec9 would fix #50? Like, what was your debug process? I reproduced on my end, and it also resulted in logs in the following form: /nix/store/2vw01kp965w65cjb390c5hpcn220qavi-source/nixos/modules//nix/store/2vw01kp965w65cjb390c5hpcn220qavi-source/nixos/modules/config/debug-info.nix. So there were two repeated modulePaths, somehow. | 19:18:30 |
Zhaofeng Li | In reply to @winterqt:nixos.dev Zhaofeng Li: How did you determine that the change in e58dde1be08cf7fbac7299e928b6d0e143edbec9 would fix #50? Like, what was your debug process? * I reproduced on my end, and it also resulted in logs in the following form: '/nix/store/2vw01kp965w65cjb390c5hpcn220qavi-source/nixos/modules//nix/store/2vw01kp965w65cjb390c5hpcn220qavi-source/nixos/modules/config/debug-info.nix': no such file or directory. So there were two repeated modulePaths, somehow. | 19:19:15 |
Zhaofeng Li | Then I added traces to the split docs build process in documentation.nix, and found out that it was trying to do a removePrefix on the full module paths using modulesPath, and remembered that I was explicitly setting modulesPath in Colmena | 19:21:04 |
Winter (she/her) | In reply to @zhaofeng:zhaofeng.li Then I added traces to the split docs build process in documentation.nix, and found out that it was trying to do a removePrefix on the full module paths using modulesPath, and remembered that I was explicitly setting modulesPath in Colmena got it -- why were you setting it in the first place, anyways? | 20:10:36 |
Zhaofeng Li | In reply to @winterqt:nixos.dev got it -- why were you setting it in the first place, anyways? It was added very early on to debug some missing argument problems that turned out to be unrelated. But yeah, shouldn't have added it in the first place. | 20:13:16 |
Winter (she/her) | got it. | 20:56:28 |
Winter (she/her) | does Colmena add much overhead to the underlying things it does? or does speed depend on build speed + ssh connection latency + multiple ssh connection handshakes (unless you use ControlMaster)? | 20:57:14 |
Zhaofeng Li | In reply to @winterqt:nixos.dev does Colmena add much overhead to the underlying things it does? or does speed depend on build speed + ssh connection latency + multiple ssh connection handshakes (unless you use ControlMaster)? At its core Colmena is basically a "shell command scheduler" for lack of a better term, and it doesn't add much latency to what it shells out to do (nix-instantiate, nix-build, ssh) | 21:03:44 |
Winter (she/her) | right, that's what i got from the code | 21:05:09 |
Zhaofeng Li | And regarding ControlMaster, it is indeed required if you have a lot of key files, for instance, otherwise the deployment can be very slow. And before you jump in suggesting using thrussh/ssh2, yes, I have considered it (and it was the plan for the beginning), but the downsides outweigh the upsides: https://github.com/zhaofengli/colmena/issues/6 | 21:07:13 |