!UKDpaKNNsBpOPfLWfX:zhaofeng.li

Colmena

50 Members
A simple, stateless NixOS deployment tool - https://github.com/zhaofengli/colmena25 Servers

Load older messages


SenderMessageTime
24 Jun 2022
@hexa:lossy.networkhexastrictly speaking: activation happens at boot time for goal=boot19:18:46
@hexa:lossy.networkhexaso don't upload post-activiation secrets in that case unless --reboot is given and then post-reboot 😄19:19:14
@hexa:lossy.networkhexa * so don't upload post-activiation secrets in that case unless --reboot is given19:19:22
@hexa:lossy.networkhexa * so don't upload post-activiation secrets in that case unless --reboot is given and then post-reboot 😄19:19:26
25 Jun 2022
@winterqt:nixos.devWinter (she/her)

So I'm trying to debug a weird issue.

home-manager revisions, for at least a few months now(???), have failed to build their nmd fetcher derivation when building for x86_64-linux with Colmena's build on target feature, when the system != x86_64-linux. The derivation in question is defined here.

HM commit e66f0ff69a6c0698b35034b842c4b68814440778 suddenly fixes the fetching of this derivation, by merely changing the commit hash. The diff between the failing and the succeeding derivations is just as you'd expect: just the hashes and revisions change, nothing else.

The error that's occurring with these faulty revisions are error: a 'x86_64-linux' with features {} is required to build '/nix/store/7fhf5jvnmz8wpy38v44j5cfn7mgxg7kj-nmd.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test} (7f... is failing in the diff). They both have system = x86_64-linux in the derivation, so what's going on?

16:55:50
@zhaofeng:zhaofeng.liZhaofeng LiAnother person [encountered the same thing](https://matrix.to/#/!BgJZHVOYkwVcEKLAyM:nixos.org/$pYUmOSAIpDOIbLl4q2TMc-0iyKrimWZfnFQi4Fdm6RA?via=zhaofeng.li&via=matrix.org&via=matrix.eno.space) in the deployment channel. It's a bit weird that it worked before.20:58:32
@zhaofeng:zhaofeng.liZhaofeng LiIs it possible that the original output was cached somehow?20:58:55
@winterqt:nixos.devWinter (she/her) That was my thought too. 21:47:06
@winterqt:nixos.devWinter (she/her)
In reply to @zhaofeng:zhaofeng.li
Is it possible that the original output was cached somehow?
* That was my thought too.
21:47:39
@winterqt:nixos.devWinter (she/her)Do you have any clue how the working version could be evaluating just fine, though? I GC'd my store before evaluating and the same results happened (I could eval one but not the other.)21:48:24
@winterqt:nixos.devWinter (she/her)Oh, I see the issue maybe.21:53:21
@winterqt:nixos.devWinter (she/her) I run home-manager on the same system, which in the right sequence of events, would cache the needed nmd source. 21:53:45
@winterqt:nixos.devWinter (she/her)Hah.21:53:49
@winterqt:nixos.devWinter (she/her)I think this is the reason why 😆21:54:15
@winterqt:nixos.devWinter (she/her)

https://github.com/nix-community/home-manager/commit/64ab7d6e8d157848ec285cd267db29e2f14c1076 switched HM to use a flake input for nmd, but it uses flake-compat to actually import it in docs/default.nix

I think this will still cause the same issue, since it really didn't change anything (as in, it still does IFD)? Correct me if I'm wrong.

22:03:52
26 Jun 2022
@zhaofeng:zhaofeng.liZhaofeng Li
In reply to @winterqt:nixos.dev

https://github.com/nix-community/home-manager/commit/64ab7d6e8d157848ec285cd267db29e2f14c1076 switched HM to use a flake input for nmd, but it uses flake-compat to actually import it in docs/default.nix

I think this will still cause the same issue, since it really didn't change anything (as in, it still does IFD)? Correct me if I'm wrong.

It should fix the issue since the dependency flake is no longer fetched in a derivation
01:48:02
@winterqt:nixos.devWinter (she/her)
In reply to @zhaofeng:zhaofeng.li
It should fix the issue since the dependency flake is no longer fetched in a derivation
Ah, flake-compat only uses builtins.
02:59:51
@zhaofeng:zhaofeng.liZhaofeng Li Wanja Hentze: Are you still interested in doing that? There's another person who wants to take on this, so I'm just confirming in order to avoid duplicating work. 07:34:12
@dantefromhell:matrix.orgdantefromhell
In reply to @huyage:matrix.org
I had no idea people use Nix for this. I used to do PXE boot. But now most IaaS bundle those 2 things together. For example, you ask for (provision) an EC2 instance running an AMI image (bootstrap). Is your use case like a local bare metal setup?
exactly my issue. And each cloud peovider does provisioning slightly different and than it wont for bare-metal. I want something that works same in all situations.
rescue-system + SSH + magic have served well for decades already 😉
12:06:44
@phaer:matrix.orgphaer

dantefromhell: Me too, I've been hacking on solutions for a while. My primary provider is hetzner (both cloud and bare-metal) and my general approach was to write a small script which collects info like hostname, available disks, network config, etc from a rescue system, then uses kexec to switch into a "live" nixos system while passing the collected info via kernel commandline and then partitioning and installing nixos from there on as normal.

My first implementation is available at https://github.com/dep-sys/nixos-zfs-installer but ended up with huge kexec images, so after learning more about nix(os) and looking into not-os in between, I restarted with https://github.com/dep-sys/nix-dabei.

That's not finished yet, but works for me and the general approach should be pretty portable between providers. Also because it should be rather trivial to build an iso or efi executable for the same expression used for kexec. same for netboot, but i haven't looked into that yet

12:48:05
@phaer:matrix.orgphaerI don't have too much time for it atm 😢 But next steps are more tests, running tests in ci, decreasing closure size (smaller kernel, less userland tools), then porting the hetzner scripts over from the old attempt while keeping the flake provider-agnostic. Always happy about feedback, questions and collaborators :)12:49:50
@dantefromhell:matrix.orgdantefromhell phaer: Yeah I've seen your `nixos-zfs-installer` repo before. Looked promising but I havent tested it. I ran into the same issue w/ RAM size & kexec. I stumbled over https://netboot.xyz and it seems to support NixOS - I guess a custom NixOS image is not too hard. Their PXE images are very small so kexec shouldnt be a problem. But than self-hosting another component increases overall tech-stack complexity. 17:59:52
@dantefromhell:matrix.orgdantefromhellYour approach to `nix-dabei` seems fairly similar to what I pondered a minimal `netboot` could look like 🤔 What would it take to get the zfs & raspberry features working (not necessarily both at the same time)? Those are hard requirements for a good solution in my environment18:02:41
@bl1nk:matrix.orgbl1nk I am currently facing an issue when installing paperless on a host with colmena. I wonder if I am pinning nixpkgs correctly, anybody able to help me out with this? My hive.nix looks like this: https://gist.github.com/bl1nk/e8f4fe66b036a660166d69fac1af30fe
The underlying issue is: https://discourse.nixos.org/t/unable-to-install-paperless-ngx/19962
18:11:15
@bl1nk:matrix.orgbl1nkMy issue was unrelated to Colmena. Thanks and sorry for the noise. 🙂22:09:17
27 Jun 2022
@phaer:matrix.orgphaer
In reply to @dantefromhell:matrix.org
Your approach to `nix-dabei` seems fairly similar to what I pondered a minimal `netboot` could look like 🤔

What would it take to get the zfs & raspberry features working (not necessarily both at the same time)? Those are hard requirements for a good solution in my environment
Not much i think, I just havent found time for it yet. ZFS is a minimal requirement for me too, so i'll probably start working on it soon. Don't expect any hiccups, might work on first try.
09:23:28
@phaer:matrix.orgphaer dantefromhell: For the raspberry, i think the right kernel and bootloader should be enough but i don't own any raspberries anymore 09:24:13
@phaer:matrix.orgphaer dantefromhell: Needless to say, I'd be happy to support if you want to give it a shot and implement either ZFS or raspberry support into nix-dabei 09:24:48
28 Jun 2022
@ma27:nicht-so.sexyma27 changed their profile picture.11:37:46
@shine:proqqul.netTaeer Bar-YamIs there a way to have colmena stop trying and exit once any one build has failed?13:28:08

There are no newer messages yet.


Back to Room ListRoom Version: 6