| 7 Feb 2024 |
 K900 | Sounds like DNS | 21:39:09 |
 netpleb | In reply to @k900:0upti.me
Sounds like DNS right, so I just thought of one thing which might fix it (i feel like i have tried everything already)...but this server actually gets its public ip via a wireguard interface, and I used wg-quick in that interface and did set the dns = [ <some server> ] option. I am going to remove that and cross my fingers! :-) | 21:40:35 |
| netpleb | hmm, sadly that did not seem to solve it | 21:44:40 |
| netpleb | what logs should i post which would help diagnose? | 21:44:54 |
| netpleb | Redacted or Malformed Event | 21:46:52 |
| K900 | Do you actually have working DNS? | 21:49:31 |
| K900 | Like, can you ping that domain name? | 21:49:39 |
| netpleb | yes, I can ping that domain name no problem | 21:50:12 |
| netpleb | [root@netpleb-public-services:~]# systemctl status acme-netpleb.com.service
○ acme-netpleb.com.service - Renew ACME certificate for netpleb.com
Loaded: loaded (/etc/systemd/system/acme-netpleb.com.service; linked; preset: enabled)
Active: inactive (dead)
TriggeredBy: ● acme-netpleb.com.timer
Feb 07 21:48:41 netpleb-public-services systemd[1]: Dependency failed for Renew ACME certificate for netpleb.com.
Feb 07 21:48:41 netpleb-public-services systemd[1]: acme-netpleb.com.service: Job acme-netpleb.com.service/start failed with result 'dependency'.
[root@netpleb-public-services:~]# ping netpleb.com
PING netpleb.com (38.45.103.128) 56(84) bytes of data.
64 bytes from ns1.netpleb.com (38.45.103.128): icmp_seq=1 ttl=64 time=0.041 ms
64 bytes from ns1.netpleb.com (38.45.103.128): icmp_seq=2 ttl=64 time=0.064 ms
^C
--- netpleb.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1064ms
rtt min/avg/max/mdev = 0.041/0.052/0.064/0.011 ms
| 21:51:06 |
| K900 | Not that | 21:51:36 |
| K900 | acme-v02.api.letsencrypt.org | 21:51:46 |
| K900 | Can you ping that? | 21:51:49 |
| netpleb | hmm, nope! wtf, I can ping google.com just fine though. What is going on? | 21:53:15 |
| K900 | You have a DNS problem | 21:55:01 |
| K900 | Have fun | 21:55:03 |
| netpleb | I am obviously not an expert in these things (though getting to know/learn Nix, both the language and the OS has been overall a rewarding experience). How is it possible that I can ping google but not letsencrypt? | 21:58:24 |
| netpleb | * ... | 21:59:26 |
| K900 | Something about your DNS config is broken | 22:01:14 |
| K900 | That's not really a NixOS problem | 22:01:23 |
| K900 | More of a general networking problem | 22:01:28 |
| netpleb | ok, sorry, i fixed the dns issue already. I am now able to ping it:
[root@netpleb-public-services:~]# ping acme-v02.api.letsencrypt.org
PING ca80a1adb12a4fbdac5ffcbc944e9a61.pacloudflare.com (172.65.32.248) 56(84) bytes of data.
64 bytes from 172.65.32.248: icmp_seq=1 ttl=59 time=94.5 ms
64 bytes from 172.65.32.248: icmp_seq=2 ttl=59 time=93.1 ms
64 bytes from 172.65.32.248: icmp_seq=3 ttl=59 time=108 ms
64 bytes from 172.65.32.248: icmp_seq=4 ttl=59 time=100 ms
| 22:02:04 |
| K900 | Now you can restart the ACME service | 22:03:14 |
| K900 | And maybe it'll actually succeed | 22:03:18 |
| netpleb | In reply to @k900:0upti.me
Now you can restart the ACME service ok, is there a "parent" acme service i should restart that will redo all of them? i have one for a subdomain and one for the tld | 22:04:03 |
| K900 | No | 22:04:24 |
| netpleb | [root@netpleb-public-services:~]# systemctl restart acme-netpleb.com.service
A dependency job for acme-netpleb.com.service failed. See 'journalctl -xe' for details.
[root@netpleb-public-services:~]# journalctl -xeu acme-netpleb.com.service
Feb 07 21:59:35 netpleb-public-services systemd[1]: Dependency failed for Renew ACME certificate for netpleb.com.
░░ Subject: A start job for unit acme-netpleb.com.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit acme-netpleb.com.service has finished with a failure.
░░
░░ The job identifier is 73 and the job result is dependency.
Feb 07 21:59:35 netpleb-public-services systemd[1]: acme-netpleb.com.service: Job acme-netpleb.com.service/start failed with result 'dependency'.
| 22:05:20 |
| netpleb | the subdomain got further along it seems:
[root@netpleb-public-services:~]# systemctl status acme-jitsi.netpleb.com
× acme-jitsi.netpleb.com.service - Renew ACME certificate for jitsi.netpleb.com
Loaded: loaded (/etc/systemd/system/acme-jitsi.netpleb.com.service; linked; preset: enabled)
Active: failed (Result: exit-code) since Wed 2024-02-07 22:06:08 UTC; 17s ago
TriggeredBy: ● acme-jitsi.netpleb.com.timer
Process: 1244 ExecStart=/nix/store/miwhrhajjh9n1pz8zlb5vywnl6qczfad-unit-script-acme-jitsi.netpleb.com-start/bin/acme-jitsi.netpleb.com-start (code=exited, status=10)
Main PID: 1244 (code=exited, status=10)
CPU: 94ms
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1247]: 2024/02/07 22:06:08 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/312750532087
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1247]: 2024/02/07 22:06:08 Could not obtain certificates:
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1247]: error: one or more domains had a problem:
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1247]: [jitsi.netpleb.com] [jitsi.netpleb.com] acme: error presenting token: rfc2136: failed to insert: DNS update failed: server replied: SERVFAIL
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1244]: + echo Failed to fetch certificates. This may mean your DNS records are set up incorrectly. Selfsigned certs are in place and dependant services will still start.
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1244]: Failed to fetch certificates. This may mean your DNS records are set up incorrectly. Selfsigned certs are in place and dependant services will still start.
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1244]: + exit 10
Feb 07 22:06:08 netpleb-public-services systemd[1]: acme-jitsi.netpleb.com.service: Main process exited, code=exited, status=10/n/a
Feb 07 22:06:08 netpleb-public-services systemd[1]: acme-jitsi.netpleb.com.service: Failed with result 'exit-code'.
Feb 07 22:06:08 netpleb-public-services systemd[1]: Failed to start Renew ACME certificate for jitsi.netpleb.com.
| 22:07:52 |
| netpleb | * the subdomain got further along it seems (also, thank you in advance for your help, I have been struggling with this for days before reaching out here):
[root@netpleb-public-services:~]# systemctl status acme-jitsi.netpleb.com
× acme-jitsi.netpleb.com.service - Renew ACME certificate for jitsi.netpleb.com
Loaded: loaded (/etc/systemd/system/acme-jitsi.netpleb.com.service; linked; preset: enabled)
Active: failed (Result: exit-code) since Wed 2024-02-07 22:06:08 UTC; 17s ago
TriggeredBy: ● acme-jitsi.netpleb.com.timer
Process: 1244 ExecStart=/nix/store/miwhrhajjh9n1pz8zlb5vywnl6qczfad-unit-script-acme-jitsi.netpleb.com-start/bin/acme-jitsi.netpleb.com-start (code=exited, status=10)
Main PID: 1244 (code=exited, status=10)
CPU: 94ms
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1247]: 2024/02/07 22:06:08 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/312750532087
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1247]: 2024/02/07 22:06:08 Could not obtain certificates:
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1247]: error: one or more domains had a problem:
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1247]: [jitsi.netpleb.com] [jitsi.netpleb.com] acme: error presenting token: rfc2136: failed to insert: DNS update failed: server replied: SERVFAIL
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1244]: + echo Failed to fetch certificates. This may mean your DNS records are set up incorrectly. Selfsigned certs are in place and dependant services will still start.
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1244]: Failed to fetch certificates. This may mean your DNS records are set up incorrectly. Selfsigned certs are in place and dependant services will still start.
Feb 07 22:06:08 netpleb-public-services acme-jitsi.netpleb.com-start[1244]: + exit 10
Feb 07 22:06:08 netpleb-public-services systemd[1]: acme-jitsi.netpleb.com.service: Main process exited, code=exited, status=10/n/a
Feb 07 22:06:08 netpleb-public-services systemd[1]: acme-jitsi.netpleb.com.service: Failed with result 'exit-code'.
Feb 07 22:06:08 netpleb-public-services systemd[1]: Failed to start Renew ACME certificate for jitsi.netpleb.com.
| 22:08:25 |
| K900 | Your DNS server said no | 22:10:12 |
| K900 | You should now go look at the logs for that | 22:10:24 |
