| 18 Sep 2025 |
 lillecarl | In reply to @markuskowa:matrix.org
If the generation of config files would be reusable (I.e. separated in the module system), we could reuse them for dinix, or container image generation. Yep, reusable across local dev and in containers, but also if someone makes a dinix module for the "LAMP" stack we could easily stick it into systemd on a VM of your choice | 13:23:30 |
| lillecarl | In reply to @kacper.uminski:matrix.org
Hi, all! I am currently trying to build a stateless cluster using Nixos. The main thing is that I want all things to be wiped on each boot. The idea is to have one management node serving an ipxe image over pxe, a kernel, and an initrd. The ipxe would pull the kernel and initrd, which would then torrent a root tarball, extract it, and kexec/switch_root into it. Does anyone have an idea where to even start with this? I did try nix-generators for the root kexec tarball, but that doesn't seem to work well when you already have a /nix in root already... Any suggestions would be most appreciated! :) Sounds like you've made a few "too many" decisions already. I couldn't help you, I would probably try to make a nixos iPXE image and make the iPXE server a nix build cache instead, build new images when you need new systemd or kernel | 13:25:57 |
| lillecarl | nixos-anywhere does the kexec thing already work to install nixos over "any" Linux distro, so you could investigate their thing.
If you wanna use nix, use nix :p torrents don't belong in Nix (even though they map well to the store immutability) | 13:27:19 |
 Julien | Maybe some simple stuff that could work is to setup syncthing on the nixos machine such that when it’s brought up it downloads P2P all the state of the machine | 13:29:39 |
 markuskowa | https://github.com/markuskowa/nix-system/blob/master/modules%2Fkea.nix
Kacper Uminski: you can generate pxe capable image directly with nixos (see my example above) | 13:30:05 |
| lillecarl | (ask in #users:nixos.org too, we're a small community) | 13:31:26 |
| lillecarl | In reply to @markuskowa:matrix.org
https://github.com/markuskowa/nix-system/blob/master/modules%2Fkea.nix
Kacper Uminski: you can generate pxe capable image directly with nixos (see my example above) I love you man! ❤️❤️😄 | 13:32:00 |
| lillecarl | In reply to @julienmalka:matrix.org
Maybe some simple stuff that could work is to setup syncthing on the nixos machine such that when it’s brought up it downloads P2P all the state of the machine Yes, let nix do the nixy bits, how you manage state is free-for-all 😁 | 13:33:14 |
 Kacper Uminski | Yes, I realize the idea is quite cursed. The main idea is to minimize stress on the control node, so a centralized cache or pxe server for the full filesystem is a no-no. All filesystems have to be reformatted on reboot, including /nix/store. We already have a setup like this working with Debian, but it's hacked together using shell-scripts, python, and preseed files. Hence I am investigating alternatives. :) | 13:35:28 |
| Kacper Uminski | Thanks! I will investigate this, at least for the minimal bootup. :) | 13:36:27 |
| lillecarl | @kacper.uminski:matrix.org: do you work for a VPN company perhaps? 😄 | 13:37:46 |
| Kacper Uminski | No, I'm just a member of Lysator (lysator.liu.se), a computer association in Linköping, and we are currently building an HPC cluster. Just as a cool tidbit, Nationella Superdatorcentrum (NSC) does the torrent-boot thing. | 13:39:00 |
| markuskowa | In reply to @kacper.uminski:matrix.org
Yes, I realize the idea is quite cursed. The main idea is to minimize stress on the control node, so a centralized cache or pxe server for the full filesystem is a no-no. All filesystems have to be reformatted on reboot, including /nix/store. We already have a setup like this working with Debian, but it's hacked together using shell-scripts, python, and preseed files. Hence I am investigating alternatives. :) I do not have the overview over your setup/requirements, but i use this PXE setup as an installer. There are many options, such as overlayfs, that you use here to customize it. | 13:40:15 |
| lillecarl | @kacper.uminski:matrix.org: Boot off iPXE, dd the image to disk, kexec, rebuild. You can have multiple cache servers and will easily be able to saturate your links with NGINX from *a few* cache servers. You can share the same signing key across them and use RR DNS for loadbalancing.
The torrent thing sounds like a solution to "immutable Debian" rather than content delivery | 13:46:19 |
| lillecarl | It'll be faster, no torrent startup time, only fetch what your node needs etc... :) | 13:50:16 |
| Kacper Uminski | Hmm, I will try this. Thank you! | 13:53:24 |
| Kacper Uminski | Just by the way, I am quite new to netbooting as a whole, so I do ask for leniency anytime I utter something unspeakably stupid. :) | 13:58:58 |
| markuskowa | Kacper Uminski: I run a small hpc cluster (20 nodes) at Fysikum, SU fully on NixOS. If you're interested i can show, how it is setup. | 15:42:05 |
| Kacper Uminski | That would be very cool! Do you have a git repo for it or something? | 15:47:29 |
| markuskowa | No, I did not publish the configuration for security reasons. | 15:49:23 |
| Kacper Uminski | Ah, yeah, that's reasonable. | 15:49:51 |
| lillecarl | https://github.com/aristanetworks/nix-serve-ng Arista Networks reimplementation of nix-serve in Haskell that's supposed to be superduperfast. Kacper Uminski 😄 | 21:37:35 |
| lillecarl | Can LIU subsidize you coming up to Stockholm to drink beer with us on the 30th? | 21:38:08 |
| Kacper Uminski | In reply to @lillecarl:matrix.org
Can LIU subsidize you coming up to Stockholm to drink beer with us on the 30th? No, sadly. I'm just a student, not an employee, and LiU is not very spend-happy right now. The only reason Lysator got to keep its server room was because we rode the AI bubble and promised to build an AI cluster. :( | 21:42:23 |
| Kacper Uminski | (For use in courses at the university.) | 21:43:02 |
| 19 Sep 2025 |
 adisbladis | That's the kind of thing I'm trying to accomplish with https://github.com/adisbladis/adios/ | 05:25:48 |
| lillecarl | Beware that COVID is running rampant right now in Stockholm guys! | 17:29:42 |
| 22 Sep 2025 |
| lillecarl changed their display name from LilleCarl (Salsa9) to lillecarl. | 10:37:33 |
| 30 Sep 2025 |
 talyz | Meetup today or no? | 08:18:46 |
| markuskowa | Yes, absolutely! | 08:19:13 |
