| 13 Dec 2022 |
 m1cr0man | 103 attempts later and not a single one has failed 😕 | 22:38:37 |
| m1cr0man | Okay so being a bit smarter with this debugging, I am looking through the build logs on Hydra for successful builds and checking where any retry logic was triggered and how many times. if you search this build https://hydra.nixos.org/build/201652934/nixlog/1 for "s_client -brief" You will see an instance in the first few matches in which it has to be retried 3 consecutive times (and works on the third). The method performing this is here: https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/acme.nix#L407-L418 and is configured for 3 retries. You can also see the webserver giving the error client closed connection while waiting for request on 2 of the 3 attempts.
I think I need to increase the delays + number of retries for this method and any others that are waiting on web responses. Even from that log, I can't see any reason the server wasn't able to respond or why the client had sent a partial request. Hopefully this will be sufficient to stop the failures.
| 22:59:25 |
| m1cr0man | Welp, here's a PR https://github.com/NixOS/nixpkgs/pull/205983 hopefully this does the trick. I have good evidence to support increased retries + delays solving the issue. | 23:44:37 |
| 14 Dec 2022 |
| m1cr0man | Awh dammit I just realized that vscode auto formatted it 🤦🤦 will fix tomorrow | 00:22:06 |
|  Alesya Huzik joined the room. | 11:13:02 |
| 21 Dec 2022 |
|  ThatsNoMoon joined the room. | 05:01:47 |
| ThatsNoMoon left the room. | 20:16:41 |
| 22 Dec 2022 |
| m1cr0man | Thanks for the merge Raito! 🙂 | 03:25:12 |
 raitobezarius | with pleasure | 04:15:18 |
| raitobezarius | thanks for all the amazing work on ACME ;) | 04:15:28 |
| 24 Dec 2022 |
 K900 | https://hydra.nixos.org/build/202864292/nixlog/5 | 07:51:42 |
| K900 | It broke again :( | 07:51:47 |
| raitobezarius | In reply to @k900:0upti.me
https://hydra.nixos.org/build/202864292/nixlog/5 did it? | 20:29:55 |
 hexa | Probably fine after restart | 21:11:52 |
| 25 Dec 2022 |
|  ahmed left the room. | 10:34:49 |
| 26 Dec 2022 |
| m1cr0man | Huh I think my Matrix client was broke, those messages from Saturday just appeared. | 14:17:22 |
| 31 Dec 2022 |
| hexa | 😲 | 19:49:12 |
| 1 Jan 2023 |
|  v0|d left the room. | 12:05:26 |
| 11 Jan 2023 |
| hexa | so I had to restore a host yesterday and I could've restored the certificates from a backup, but didn't | 15:55:52 |
| hexa | the acme module recreated all certs no problemo | 15:56:04 |
| hexa | on the first try | 15:56:08 |
| hexa | mind you, the host has 10 different ones 🙂 | 15:56:25 |
| hexa | 🙏 | 15:56:38 |
 Arian | yay | 15:56:47 |
| hexa | our ACME story is truly great | 15:57:10 |
| 12 Jan 2023 |
| raitobezarius | I also had situations like this and really it's pure joy | 10:22:48 |
| 13 Jan 2023 |
| m1cr0man | Heh so, the reason I took developing of the wildcard cert support years ago was because I was deploying about 30 domains to a couple of servers, one of which had a subdomain per user (it was for a network society). All I know is, when I check those domains 4 years (holy shit time flies) later, they still work ;) | 18:46:03 |
| m1cr0man | In reply to @hexa:lossy.network
mind you, the host has 10 different ones 🙂 Btw this is why we implemented credential sharing across multiple instances of the renewal service. There's a 5 accounts per day rate limit | 20:08:49 |
| m1cr0man | when I get some motivation I really want to port some of the features of the renewal script directly to lego. There's stuff in there that would be genuinely easier in the tool itself | 20:09:21 |
| hexa | good luck with that | 20:10:12 |
