!MthpOIxqJhTgrMNxDS:nixos.org

NixOS ACME / LetsEncrypt

103 Members
Another day, another cert renewal44 Servers

Load older messages


SenderMessageTime
29 Nov 2024
@hexa:lossy.networkhexaworked on the third try on hydra02:34:51
@thinkchaos:matrix.orgThinkChaos

It's not stuck on "Releasing lock", that process has exited: Main PID: 180885 (code=exited, status=0\/SUCCESS)
The code actually does nothing after printing that, the script just exits which automatically frees the lock (source)

Based on the CGroup content I think it's stuck on reloading Nginx though I don't understand how that would block or why it's doing that, as Nginx is supposed to reload itself through nginx-config-reload.service.
What's the content of /nix/store/vw98jrrshrz371az32ssbcwrr3bz2fqs-acme-postrun?
What's the value of services.nginx.enableReload? Did you add nginx to the cert's reloadServices?

03:23:40
@hexa:lossy.networkhexa
#!/nix/store/0irlcqx2n3qm6b1pc9rsd2i8qpvcccaj-bash-5.2p37/bin/bash
cd /var/lib/acme/spam.lossy.network
if [ -e renewed ]; then
  rm renewed
  systemctl reload nginx

  
fi
12:38:15
@hexa:lossy.networkhexa *
#!/nix/store/0irlcqx2n3qm6b1pc9rsd2i8qpvcccaj-bash-5.2p37/bin/bash
cd /var/lib/acme/spam.lossy.network
if [ -e renewed ]; then
  rm renewed
  systemctl reload nginx # <-- stuck here
fi
12:38:32
@hexa:lossy.networkhexa I have not set enableReload 12:39:10
@hexa:lossy.networkhexa
  security.acme.certs."spam.${config.networking.domain}" = {
    postRun = ''
      systemctl reload nginx
    '';
  };
12:40:03
@hexa:lossy.networkhexa converting that to reloadServices is obviously WIP 12:40:46
@thinkchaos:matrix.orgThinkChaos I think you should set enableReload = true and remove all your custom reloading logic. The Nginx module will handle it: https://github.com/NixOS/nixpkgs/blob/0c582677378f2d9ffcb01490af2f2c678dcb29d3/nixos/modules/services/web-servers/nginx/default.nix#L1317-L1342 14:39:11
@hexa:lossy.networkhexaok, cool18:58:37
@hexa:lossy.networkhexa doesn't explain why systemctl reload nginx gets stuck 😄 18:58:45
@hexa:lossy.networkhexa
Thread 1 (Thread 0x7f7f4c1c5680 (LWP 180909) "systemctl"):
#0  0x00007f7f4c50963c in ppoll () from target:/nix/store/pacbfvpzqz2mksby36awvbcn051zcji3-glibc-2.40-36/lib/libc.so.6
No symbol table info available.
#1  0x00007f7f4c82270b in ppoll_usec () from target:/nix/store/ivqjhj99firnjq7gp14qf35821viwi5m-systemd-256.7/lib/systemd/libsystemd-shared-256.so
No symbol table info available.
#2  0x00007f7f4c89e33a in bus_poll () from target:/nix/store/ivqjhj99firnjq7gp14qf35821viwi5m-systemd-256.7/lib/systemd/libsystemd-shared-256.so
No symbol table info available.
#3  0x00007f7f4c89e6c5 in sd_bus_wait () from target:/nix/store/ivqjhj99firnjq7gp14qf35821viwi5m-systemd-256.7/lib/systemd/libsystemd-shared-256.so
No symbol table info available.
#4  0x00007f7f4c6c41b9 in bus_wait_for_jobs () from target:/nix/store/ivqjhj99firnjq7gp14qf35821viwi5m-systemd-256.7/lib/systemd/libsystemd-shared-256.so
No symbol table info available.
#5  0x00005641d8690e2c in verb_start ()
No symbol table info available.
#6  0x00005641d8672bea in main ()
18:59:54
@thinkchaos:matrix.orgThinkChaos You could run the service's ExecReload manually to see if it's there or in Systemd it's hanging 19:11:11
@thinkchaos:matrix.orgThinkChaosIt only does 2 things: check the config, and send a SIGHUP19:11:58
@hexa:lossy.networkhexasystemctl reload nginx blocks, I think I established that earlier19:16:32
@hexa:lossy.networkhexauhh, sorry19:16:37
@hexa:lossy.networkhexaI mean I established that the both work individually19:16:56
@hexa:lossy.networkhexait is systemctl reload that is stuck for some reason19:17:03
@hexa:lossy.networkhexahttps://gist.github.com/mweinelt/f099ec270ace7cb197954e23871471be19:21:08
@admin:nixos.org@admin:nixos.org joined the room.19:22:24
@admin:nixos.org@admin:nixos.org left the room.19:22:37
@thinkchaos:matrix.orgThinkChaos Respectfully, I don't want to spend more time investigating this issue since it's in your personal config and not the NixOS modules.
Your strace ends with ask-password related stuff so it's likely waiting to authenticate somehow.
If you switch to reloadServices it uses --no-block.
And better yet, if you switch to enableReload you'll use the battle tested solution.
19:54:56
1 Dec 2024
@m1cr0man:m1cr0man.comm1cr0manI have another "fun" set of upstreaming work completed. I estimate this one at half the chance of being merged than the previous change, simply because of the structure of lego's cmd code + error handling. https://github.com/go-acme/lego/compare/master...m1cr0man:lego:renew-rc-2 https://github.com/m1cr0man/nixpkgs/commit/53846b07f5037e854993366beab3e0a618d1fd68 I have not opened PRs yet, will do that in a second01:52:09
@m1cr0man:m1cr0man.comm1cr0manWith this work, I think the ACME module is in one of the best states it has ever been in. The remaining bash scripting in the module does only 2 things primarily: 1. Perform simple file operations like cp, chmod, chown. 2. Handle concurrency limits. The latter is being looked into by ThinkChaos too, see earlier discussions :) 02:00:32
@m1cr0man:m1cr0man.comm1cr0manLol, that ended quickly https://github.com/go-acme/lego/pull/236602:18:10
@m1cr0man:m1cr0man.comm1cr0manhttps://github.com/go-acme/lego/issues/2367 🤷 lets hope it doesn't take years02:37:50
5 Dec 2024
@maka_77x:matrix.orgmaka_77x joined the room.01:53:01
16 Dec 2024
@k900:0upti.meK900So uh 23:41:24
@k900:0upti.meK900Do we have anything that can at least paper over the ordering issues 23:41:42
@k900:0upti.meK900Without making things even more complicated 23:41:51
@k900:0upti.meK900Because the tests are flaking a lot and it's getting on my nerves 23:42:04

Show newer messages


Back to Room ListRoom Version: 6