!UUqahLbShAYkkrXmKs:matrix.org

DevOS

36 Members
Seeking help and geeking out together on https://github.com/divnix/devos & https://github.com/divnix/digga10 Servers

Load older messages


SenderMessageTime
30 Jul 2021
@kraftnix:matrix.org@kraftnix:matrix.org anyway on another topic, has anyone had issues with nix flake check when you have a lot of hosts? I've found the python vm test coredumping more as I've added more hosts (and will just pass if I rerun the twice with half the hosts each), systemd-coredump points to the pid of one of the python tests and contains systemd-coredump[134485]: Resource limits disable core dumping for process 123011 (python3). 21:31:15
@kraftnix:matrix.org@kraftnix:matrix.org *

anyway on another topic, has anyone had issues with nix flake check when you have a lot of hosts? I've found the python vm test coredumping more as I've added more hosts (and will just pass if I rerun them twice with half the hosts each), systemd-coredump points to the pid of one of the python tests and contains systemd-coredump[134485]: Resource limits disable core dumping for process 123011 (python3).

i'm guessing theres some resource limits, but i'm not sure where they might be of for what service (perhaps nix daemon)?

21:32:36
@kraftnix:matrix.org@kraftnix:matrix.org *

anyway on another topic, has anyone had issues with nix flake check when you have a lot of hosts? I've found the python vm test coredumping more as I've added more hosts (and will just pass if I rerun them twice with half the hosts each), systemd-coredump points to the pid of one of the python tests and contains systemd-coredump[134485]: Resource limits disable core dumping for process 123011 (python3).

i'm guessing theres some resource limits, but i'm not sure where they might be or for what service (perhaps nix daemon)?

21:32:47
@gtrunsec:matrix.org@gtrunsec:matrix.orgimage.png
Download image.png
22:03:27
@kraftnix:matrix.org@kraftnix:matrix.org hmm, that's per vm-test though right? for me it's the digga profilesTest that is failing, i can run them in isolation or a couple at a time but not all of them (non-deterministically failing), i can actually run them all sometimes. 22:05:56
@kraftnix:matrix.org@kraftnix:matrix.org

In terms of overall system usage, I have many 10s of GB RAM free and no
noticeable strain on CPU or anything. I've been continuously running
tests (with minor changes to ensure all systems rebuild everytime + non-cached) and it's quite non-deterministic :(

I'm starting to think it could be the test-driver.py, I see it using exec() and join() which are red flags to me from some python multiprocessing i've done

22:15:03
@gtrunsec:matrix.org@gtrunsec:matrix.orgplease, adding the virtual reosurce to here. https://github.com/divnix/digga/blob/main/src/tests.nix#L38 (and create the new branch for this test). then run it again22:18:40
@kraftnix:matrix.org@kraftnix:matrix.orgyeah, I'll give it ago, currently I'm having issues reproducing the error, it's 1 in 4 times with all hosts at the moment22:19:42
@kraftnix:matrix.org@kraftnix:matrix.orgwhich is a pain to test as it takes ~4mins to pass/fail each time. given that I literally just got it failing again I'll do the above22:20:27
@kraftnix:matrix.org@kraftnix:matrix.org running with memorySize = 4096; and cores = 4; 22:23:03
@kraftnix:matrix.org@kraftnix:matrix.org * running with memorySize = 4096; and cores = 4;, let's see how it goes 22:23:19
@gtrunsec:matrix.org@gtrunsec:matrix.org Notice: memorySize = 4096; The volume depends on how many services you will be running. 22:27:26
@kraftnix:matrix.org@kraftnix:matrix.org

fails with 4096 as well, the accompanying python error (which I think I forgot to mention earlier is:

       > Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedWriter name=''> at interpreter shutdown, possibly due to daemon threads
       > Python runtime state: finalizing (tstate=0x434b00)
       >
       > Current thread 0x00007ffff77ec340 (most recent call first):
       > 
       > /nix/store/bmxafbqrlvcwpcwd21irmirid4rkpank-stdenv-linux/setup: line 1286:     7 Aborted                 (core dumped) LOGFILE=/dev/null tests='exec(os.environ["testScript"])' /nix/store/va2yrwawaqbk676s0a0j9vcr8fj3bijf-nixos-test-driver-profiles/bin/nixos-test-driver
22:29:53
@gtrunsec:matrix.org@gtrunsec:matrix.orghttps://stackoverflow.com/questions/45267439/fatal-python-error-and-bufferedwriter22:34:59
@kraftnix:matrix.org@kraftnix:matrix.org this looks like the reason why i always use ThreadPoolExecutor with python multi-processing as it cleans itself a lot better, i'm now testing with a test-driver that does an os.exit(0) at the of the script as suggested by the post you linked, i was about to rewrite the test to use an ThreadPoolExecutor anyway 22:42:17
@kraftnix:matrix.org@kraftnix:matrix.orgyep, so one of the VMs is not exiting fast enough, when adding os.exit(0) at the end of the test-driver, the test seems to hang after finishing for an extra 2mins, and then the test-driver process finally ends and the flake check passes22:45:35
@kraftnix:matrix.org@kraftnix:matrix.orggiven the non-deterministic nature of the issue i'll repeat the build again quite a few times until i'm sure its resolved22:46:26
@timdeh:matrix.org@timdeh:matrix.org
In reply to @d4hines:matrix.org
On the topic of escape hatches and onborading, do you think I could get a lot of the benefit of devos by basically using it as a way of interacting with home-manager on, say, an Arch linux installation?
I did this once in an ad-hoc fashion. I never pushed any code, but I had an old Arch box that that I didn't have physical access to, to install NixOS, so I used the DevOS home-manager feature and some trickery to pull values from my NixOS config, such as much zshrc and tmux.conf directly into my home-manager profile unmodified. I even got my exact same xmonad config working by simply dumping it into the proper place.
22:51:03
@timdeh:matrix.org@timdeh:matrix.org
In reply to @d4hines:matrix.org
On the topic of escape hatches and onborading, do you think I could get a lot of the benefit of devos by basically using it as a way of interacting with home-manager on, say, an Arch linux installation?
* I did this once in an ad-hoc fashion. I never pushed any code, but I had an old Arch box that that I didn't have physical access to, to install NixOS, so I used the DevOS home-manager feature and some trickery to pull values from my NixOS config, such as my zshrc and tmux.conf directly into my home-manager profile unmodified. I even got my exact same xmonad config working by simply dumping it into the proper place.
22:51:21
@timdeh:matrix.org@timdeh:matrix.org * I did this once in an ad-hoc fashion. I never pushed any code, but I had an old Arch box that that I didn't have physical access to install NixOS on, so I used the DevOS home-manager feature and some trickery to pull values from my NixOS config, such as my zshrc and tmux.conf directly into my home-manager profile unmodified. I even got my exact same xmonad config working by simply dumping it into the proper place.22:52:51
@kraftnix:matrix.org@kraftnix:matrix.orgI also used home-manager + nix on arch before I moved to NixOS, but I was using classical nix and was very new to nix back them, my home-manager config was an extremely simple modification of my old dotfiles repo, so i was wondering how easy devos will make the whole experience. it's good to hear you got xmonad working, makes me want to try this more so i can red pill some of my friends22:52:58
@kraftnix:matrix.org@kraftnix:matrix.org
In reply to @kraftnix:matrix.org
given the non-deterministic nature of the issue i'll repeat the build again quite a few times until i'm sure its resolved
so i'm pretty sure it's all working now, it ran 10 times with no failures, thanks gtrunsec for the help 😀
23:25:31
31 Jul 2021
@princemachiavelli:matrix.orgprincemachiavelli joined the room.02:16:48
@blaggacao:matrix.orgDavid Arnold (blaggacao) kraftnix: w.r.t. test-driver.py: let me know what you've concluded. I'm currently doing a refactoring ("over the eons") with tfc to get a better testing experience. Any insights will be really helpful. w.r.t. home-manager, there are now "portable" profiles that work completely without setting up any nixos hosts at all. This addition makes the use case easier where you want to start onboarding all-out with hm but without any NixOS. Here is the implementation in digga 02:56:20
@yusdacra:nixos.devyusdacra changed their profile picture.17:17:58
@timdeh:matrix.org@timdeh:matrix.org I can't remember... did we ever decide to export profiles, so one can use another devoser's profiles from their own configuration? 23:47:55
@timdeh:matrix.org@timdeh:matrix.org David Arnold: Pacman99 ? 23:50:18
@timdeh:matrix.org@timdeh:matrix.org * David Arnold: Pacman99 ? ^^ 23:50:28
1 Aug 2021
@blaggacao:matrix.orgDavid Arnold (blaggacao)No, only modules or whole configs.01:10:27
@blaggacao:matrix.orgDavid Arnold (blaggacao) I'm not sure if sharing profiles would be really a good idea, in theory. Modules are the "generics" and profiles the concrete implementation. I find it to be a good balance between convenience and responsibility to pass "ownership" of any concrete implementation to the user. 01:12:49

Show newer messages


Back to Room ListRoom Version: 6