We have some expressway-e servers that I had our SSL signed using the ACME service. The Cert is about to expire and has been trying to renew the cert for several weeks and even manually clicked on Sign CSR with ACME service. Each time we get the nonce error. I see the 443 traffic out from the server and see the check on port 80 back in. Any solutions?
Below is the log output. All DNS records are in place.
Logs analysis
+++++++++++
Authorization failed as challenges given did not resolve
2021-06-16T00:05:30.879-04:00 dr-cm-exe01 management: UTCTime="2021-06-16 04:05:30,878" Module="developer.management.acmecertbotutils" Level="ERROR" CodeLocation="acmecertbotutils(85)" Detail="Certbot StdErr< abnormally: Traceback (most recent call last): File "/bin/certbot", line 11, in load_entry_point('certbot==0.36.0', 'console_scripts', 'certbot')() File "/lib64/python2.7/site-packages/certbot/main.py", line 1381, in main File "/lib64/python2.7/site-packages/certbot/main.py", line 1251, in certonly File "/lib64/python2.7/site-packages/certbot/main.py", line 1171, in _csr_get_and_save_cert File "/lib64/python2.7/site-packages/certbot/client.py", line 294, in obtain_certificate_from_csr File "/lib64/python2.7/site-packages/certbot/client.py", line 385, in _get_order_and_authorizations File "/lib64/python2.7/site-packages/certbot/auth_handler.py", line 90, in handle_authorizations File "/lib64/python2.7/site-packages/certbot/auth_handler.py", line 154, in _poll_authorizations AuthorizationError: Some challenges have failed. Some challenges have failed. >EndStdErr"
Challenges that failed
2021-06-16T00:05:30.879-04:00 dr-cm-exe01 management: UTCTime="2021-06-16 04:05:30,878" Module="developer.management.acmecertbotutils" Level="ERROR" CodeLocation="acmecertbotutils(86)" Detail="Certbot StdOut< IMPORTANT NOTES: - The following errors were reported by the server: Domain: dr-cm-exe01.medcost.com Type: connection Detail: Fetching http://dr-cm-exe01.medcost.com/.well-known/acme-challenge/SbCVA5GpdaEI9nPEBHSNw4D_9TlrEp_Xi742Ib261j8: Connection refused Domain: mra2.medcost.com Type: connection Detail: Fetching http://mra2.medcost.com/.well-known/acme-challenge/FImpixG_kXnDnBRNc6ihhh8imhv9uvBdgmS5DmmYJKA: Connection refused To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address. Additionally, please check that your computer has a publicly routable IP address and that no firewalls are preventing the server from communicating with the client. If you're using the webroot plugin, you should also verify that you are serving files from the webroot path you provided. >EndStdOut"
2021-06-16T00:05:30.903-04:00 dr-cm-exe01 management: UTCTime="2021-06-16 04:05:30,903" Module="developer.management.acmecertbotutils" Level="DEBUG" CodeLocation="acmecertbotutils(208)" Detail="Matched groups: ('urn:ietf:params:acme:error:badNonce', 'ietf:params:', 'JWS has an invalid anti-replay nonce: \"0004-TxYdcGQy60Ik7r_WwaDZWKSjRmMB_vX-cKlMYDTB0Y\"')"
Gives the error that cannot sign the CSR and displays the error
2021-06-16T00:05:30.907-04:00 dr-cm-exe01 management: UTCTime="2021-06-16 04:05:30,907" Module="developer.management.acmestate" Level="DEBUG" CodeLocation="acmestate(313)" Detail="update_acme_state: domain state {'last_sign_sub_error_id': '', 'last_sign_error_id': 'CB_BAD_NONCE', 'last_sign_error_text': 'The client sent an unacceptable anti-replay nonce', 'last_sign_datetime': '2021-06-16 00:05:30', 'last_sign_error_details': ''}"
2021-06-16T00:05:31.187-04:00 dr-cm-exe01 management: Level="ERROR" Detail="Acme Sign failed" Domain="Server", Reason="The client sent an unacceptable anti-replay nonce", ErrorCode="500" UTCTime="2021-06-16 04:05:31,187"
2021-06-16T00:05:31.187-04:00 dr-cm-exe01 management: UTCTime="2021-06-16 04:05:31,186" Module="developer.management.acmesigncommand" Level="ERROR" CodeLocation="acmesigncommand(199)" Detail="Acme Sign Command Failed: The client sent an unacceptable anti-replay nonce" Code="500"
+++++++++++++++++++++++++++++++