!MthpOIxqJhTgrMNxDS:nixos.org

NixOS ACME / LetsEncrypt

107 Members
Another day, another cert renewal45 Servers

Load older messages

SenderMessageTime
18 Nov 2024
@m1cr0man:m1cr0man.comm1cr0man

Here's the would-be change to switch to it on the module: https://github.com/m1cr0man/nixpkgs/commit/0e84622dff2ab5c8ba37cf49a3dd212b253664ee

What's nice is that this wont trigger a mass renewal on all nixos systems. domains thankfully won't part of the directory tree hashes that we have (e.g. account hash)

23:38:32
@m1cr0man:m1cr0man.comm1cr0man *

Here's the would-be change to switch to it on the module: https://github.com/m1cr0man/nixpkgs/commit/0e84622dff2ab5c8ba37cf49a3dd212b253664ee

What's nice is that this wont trigger a mass renewal on all nixos systems. Domains options are thankfully aren't part of the directory tree hashes that we have (e.g. account hash)

23:39:03
@m1cr0man:m1cr0man.comm1cr0man *

Here's the would-be change to switch to it on the module: https://github.com/m1cr0man/nixpkgs/commit/0e84622dff2ab5c8ba37cf49a3dd212b253664ee

What's nice is that this wont trigger a mass renewal on all nixos systems. Domains options are thankfully not part of the directory tree hashes that we have (e.g. account hash)

23:39:09
@m1cr0man:m1cr0man.comm1cr0manyeah my typing has gone, I have to sleep ;)23:39:21
@thinkchaos:matrix.orgThinkChaos Seems pretty straightforward, I'd just name it more explicitly, maybe --replace-cert-domains
Goon night 😉 		23:40:41
19 Nov 2024
@m1cr0man:m1cr0man.comm1cr0manAh I do like replace more than overwrite. 09:00:53
20 Nov 2024
@inayet:matrix.orgInayet removed their profile picture.00:59:18
@kamillaova:matrix.orgKamilla 'ova joined the room.12:55:49
@m1cr0man:m1cr0man.comm1cr0manA day late but I did change it 😪22:31:40
27 Nov 2024
@hexa:lossy.networkhexawith 24.11 I see acme units stuck at "Releasing lock" a bunch01:49:49
@hexa:lossy.networkhexa 
● acme-lossy.network.service                                                                  loaded activating start-post start  Renew ACME certificate for lossy.network
● acme-musique.lossy.network.service                                                          loaded activating start-post start  Renew ACME certificate for musique.lossy.network
● acme-paste.lossy.network.service                                                            loaded activating start-post start  Renew ACME certificate for paste.lossy.network
01:49:52
@hexa:lossy.networkhexa 
● acme-lossy.network.service - Renew ACME certificate for lossy.network
     Loaded: loaded (/etc/systemd/system/acme-lossy.network.service; enabled; preset: ignored)
     Active: activating (start-post) since Mon 2024-11-25 16:53:59 UTC; 1 day 8h ago
 Invocation: d25d36d6c8c04cd7886c1d4bc8d53792
TriggeredBy: ● acme-lossy.network.timer
   Main PID: 130913 (code=exited, status=0/SUCCESS); Control PID: 130934 (kia1z8g0zv7w2nd)
         IP: 0B in, 0B out
         IO: 336K read, 8K written
      Tasks: 2 (limit: 4553)
     Memory: 1.7M (peak: 19.6M)
        CPU: 510ms
     CGroup: /system.slice/acme-lossy.network.service
             ├─130934 /nix/store/0irlcqx2n3qm6b1pc9rsd2i8qpvcccaj-bash-5.2p37/bin/bash /nix/store/kia1z8g0zv7w2ndbr6bf88ybgacjldi1-acme-postrun
             └─130936 systemctl reload nginx
01:50:23
@hexa:lossy.networkhexa 
Nov 25 16:53:59 juno systemd[1]: Starting Renew ACME certificate for lossy.network...
Nov 25 16:53:59 juno acme-lossy.network-start[130913]: Waiting to acquire lock /run/acme/4.lock
Nov 25 16:53:59 juno acme-lossy.network-start[130913]: Acquired lock /run/acme/4.lock
Nov 25 16:53:59 juno acme-lossy.network-start[130913]: + set -euo pipefail
Nov 25 16:53:59 juno acme-lossy.network-start[130913]: + echo 2692063bb972c5e7df2e
Nov 25 16:53:59 juno acme-lossy.network-start[130913]: + cmp -s domainhash.txt certificates/domainhash.txt
Nov 25 16:53:59 juno acme-lossy.network-start[130913]: + '[' -e certificates/lossy.network.key ']'
Nov 25 16:53:59 juno acme-lossy.network-start[130913]: + '[' -e certificates/lossy.network.crt ']'
Nov 25 16:53:59 juno acme-lossy.network-start[130916]: ++ find accounts -name hexa@darmstadt.ccc.de.key
Nov 25 16:53:59 juno acme-lossy.network-start[130913]: + '[' -n accounts/acme-v02.api.letsencrypt.org/hexa@darmstadt.ccc.de/keys/hexa@darmstadt.ccc.de.key ']'
Nov 25 16:53:59 juno acme-lossy.network-start[130913]: + lego --accept-tos --path . -d lossy.network --email hexa@darmstadt.ccc.de --key-type ec384 --dns rfc2136 --server https://acme-v02.api.letsencrypt.org/directory renew --no-random-sleep --days 30
Nov 25 16:54:00 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:00 [INFO] [lossy.network] acme: Trying renewal with 743 hours remaining
Nov 25 16:54:00 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:00 [INFO] [lossy.network] acme: Obtaining bundled SAN certificate
Nov 25 16:54:00 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:00 [INFO] [lossy.network] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz/1756599672/435404342337
Nov 25 16:54:00 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:00 [INFO] [lossy.network] acme: Could not find solver for: tls-alpn-01
Nov 25 16:54:00 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:00 [INFO] [lossy.network] acme: Could not find solver for: http-01
Nov 25 16:54:00 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:00 [INFO] [lossy.network] acme: use dns-01 solver
Nov 25 16:54:00 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:00 [INFO] [lossy.network] acme: Preparing to solve DNS-01
Nov 25 16:54:00 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:00 [INFO] Found CNAME entry for "_acme-challenge.lossy.network.": "lossy.network._acme.lossy.network."
Nov 25 16:54:01 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:01 [INFO] [lossy.network] acme: Trying to solve DNS-01
Nov 25 16:54:01 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:01 [INFO] Found CNAME entry for "_acme-challenge.lossy.network.": "lossy.network._acme.lossy.network."
Nov 25 16:54:01 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:01 [INFO] [lossy.network] acme: Checking DNS record propagation. [nameservers=127.0.0.1:53,[::1]:53]
Nov 25 16:54:03 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:03 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
Nov 25 16:54:09 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:09 [INFO] [lossy.network] The server validated our request
Nov 25 16:54:09 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:09 [INFO] [lossy.network] acme: Cleaning DNS-01 challenge
Nov 25 16:54:09 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:09 [INFO] Found CNAME entry for "_acme-challenge.lossy.network.": "lossy.network._acme.lossy.network."
Nov 25 16:54:09 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:09 [INFO] [lossy.network] acme: Validations succeeded; requesting certificates
Nov 25 16:54:11 juno acme-lossy.network-start[130917]: 2024/11/25 16:54:11 [INFO] [lossy.network] Server responded with a certificate.
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + mv domainhash.txt certificates/
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + chown acme:acme certificates/domainhash.txt certificates/lossy.network.crt certificates/lossy.network.issuer.crt certificates/lossy.network.json certificates/lossy.network.key
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + cmp -s certificates/lossy.network.crt out/fullchain.pem
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + touch out/renewed
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + echo Installing new certificate
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: Installing new certificate
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + cp -vp certificates/lossy.network.crt out/fullchain.pem
Nov 25 16:54:11 juno acme-lossy.network-start[130927]: 'certificates/lossy.network.crt' -> 'out/fullchain.pem'
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + cp -vp certificates/lossy.network.key out/key.pem
Nov 25 16:54:11 juno acme-lossy.network-start[130928]: 'certificates/lossy.network.key' -> 'out/key.pem'
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + cp -vp certificates/lossy.network.issuer.crt out/chain.pem
Nov 25 16:54:11 juno acme-lossy.network-start[130929]: 'certificates/lossy.network.issuer.crt' -> 'out/chain.pem'
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + ln -sf fullchain.pem out/cert.pem
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + cat out/key.pem out/fullchain.pem
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + chmod 640 out/cert.pem out/chain.pem out/fullchain.pem out/full.pem out/key.pem out/renewed
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: + echo 'Releasing lock /run/acme/4.lock'
Nov 25 16:54:11 juno acme-lossy.network-start[130913]: Releasing lock /run/acme/4.lock
01:50:47
@hexa:lossy.networkhexaugh, systemctl reload nginx is blocking01:51:06
@hexa:lossy.networkhexa so all three acme units are stuck at systemctl reload nginx 01:52:06
@hexa:lossy.networkhexawhich maps to01:52:18
@hexa:lossy.networkhexa 
ExecReload=/nix/store/m489ix7hrxznh7a5fmdsijdlq4x6p5nn-nginx-1.26.2/bin/nginx -c '/nix/store/gnq12cc681ydl6l6bxkh4wrpy1vcan6x-nginx.conf' -t
ExecReload=/nix/store/k48bha2fjqzarg52picsdfwlqx75aqbb-coreutils-9.5/bin/kill -HUP $MAINPID
01:52:21
@hexa:lossy.networkhexaL1 the config test works01:52:34
@hexa:lossy.networkhexa and kill -HUP $MAINPID also works 🤔 01:53:04
@hexa:lossy.networkhexaas root01:53:14
@hexa:lossy.networkhexaacme post hooks also run as root01:53:46
@hexa:lossy.networkhexa 
root      130934  0.0  0.0   6620  3328 ?        Ss   Nov25   0:00 /nix/store/0irlcqx2n3qm6b1pc9rsd2i8qpvcccaj-bash-5.2p37/bin/bash /nix/store/kia1z8g0zv7w2ndbr6bf88ybgacjldi1-acme-postrun
root      130936  0.0  0.1  18628  7424 ?        S    Nov25   0:00  \_ systemctl reload nginx
root      131100  0.0  0.0   6620  3328 ?        Ss   Nov25   0:00 /nix/store/0irlcqx2n3qm6b1pc9rsd2i8qpvcccaj-bash-5.2p37/bin/bash /nix/store/x8pmg6g602b92rrbapxpcmb695n811lb-acme-postrun
root      131103  0.0  0.1  18628  7552 ?        S    Nov25   0:00  \_ systemctl reload nginx
root      143742  0.0  0.0   6620  3328 ?        Ss   Nov26   0:00 /nix/store/0irlcqx2n3qm6b1pc9rsd2i8qpvcccaj-bash-5.2p37/bin/bash /nix/store/kr92xf7yisa0236ls1gmacqwapc3zqxz-acme-postrun
root      143745  0.0  0.1  18628  7424 ?        S    Nov26   0:00  \_ systemctl reload nginx
01:53:50
@hexa:lossy.networkhexa 
#!/nix/store/0irlcqx2n3qm6b1pc9rsd2i8qpvcccaj-bash-5.2p37/bin/bash
cd /var/lib/acme/lossy.network
if [ -e renewed ]; then
  rm renewed
  systemctl reload nginx

  
fi
01:54:14
@hexa:lossy.networkhexarenewed does not exist anymore01:54:21
@hexa:lossy.networkhexarebooted the machine and reloading works again02:02:59
@hexa:lossy.networkhexajust wrote this down so maybe if someone else hits it we'll know it is a recurring thing? 😄 02:03:15
28 Nov 2024
@hexa:lossy.networkhexaok, just saw it on the next host 🫠01:40:56
@m1cr0man:m1cr0man.comm1cr0manthat's... weird22:01:25
@m1cr0man:m1cr0man.comm1cr0mandoes the nginx service show one of the ExecReload processes as running?22:01:49
@m1cr0man:m1cr0man.comm1cr0man I got the --replace-cert-domains --overwrite-domains --force-cert-domains PR merged to lego 😄 once it has been published in a released version, and the setup process refactor has merged, I have a patch set ready to remove the domain hash entirely (test suite remains unchanged + passes, and this won't trigger mass renewal). 22:12:03

Show newer messages

Back to Room ListRoom Version: 6