Plesk SSL Let's Encrypt Error

devahaminapurnama · October 10, 2022, 2:40am

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: test.rafik.sch.id

My web server is (include version): reverse proxy nginx-apache
nginx version: nginx/1.20.2
Apache/2.4.6 (CloudLinux)

The operating system my web server runs on is (include version):
CloudLinux 7.9 x86_64

My hosting provider, if applicable, is: Plesk Obsidian 18.0.46.2

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): Plesk Obsidian 18.0.46.2

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): Using SSL It

Hello Everyone!

More than 3 months i've try to issue domain in Plesk Panel using SSL It! well-known as Let's Encrypt, but I've got trouble like: firewall, lookup AAAA records instead of A Record. I don't use IPv6. Domain I was try is: .COM, .ORG, .SCH.ID, .ID.

I'm try following to make sure that firewall already turn off, and port 80 and 443 opened. Still can't issue Let's Encrypt. A record and Nameserver already pointed on our server. You can check on attachment as image.

I've got response from Plesk development team.

It was verified that:

Let's Encrypt is working
The latest versions of the SSL It! and Let's Encrypt extensions are installed on the server
DNS server is operational. No errors were found in regards to the domain test.example.sch.id in BIND logs on Plesk server
Subdomain sub.example.sch.id is resolving correctly
No similar cases reported to us previously.

Additionally, from Let's Encrypt debug you could see that domain is resolved but it failed on step LetsEncryptStaging:

Let's Encrypt using libunbound library for checks. libunbound is located on Let's Encrypt / Let's Debug servers. Not on the Plesk server.

The library requests all DNS servers in a chain, starting from root servers. For domain example.com DNS resolution works in the following way:

It starts from "root" servers, to get who is responsible for ".sch.id" zone
then ".sch.id" is resolved from servers responsible for ".sch.id" zone
then "example.sch.id" is resolved from servers responsible for "sch.id"
then "sub.example.sch.id" is resolved from servers responsible for "example.sch.id"

DNS resolution can fail at any stage, including ".id" zone servers, you could see it in the attached log.

Network stability for UDP/TCP connections from Let's Encrypt servers to DNS servers (root DNS servers, ".id" zone server, "sch.id" zone server, your server) also matters. Troubleshooting of the network should be performed by the server administrator.

MikeMcQ · October 10, 2022, 3:16am

Welcome to the community @devahaminapurnama
Thanks for the detailed report.

I also see the Let's Encrypt (LE) servers are failing the DNS lookups. But, you will need to wait for a better DNS expert to give better advice.

I just want to clear up a couple things

One, an AAAA record is not required. But, the LE servers look for one and must see a valid record or a proper "not found" response. If they instead timeout or get a SERVFAIL that is a problem. The same for a CAA record. It is not required but a lookup cannot timeout or get SERVFAIL.

You created a "letsdebug-test" file so that the first test request from Let's Debug site gets a 200 OK. I can also see that file from my test server. I don't see any problems using HTTP or HTTPS to reach your site (except the self-signed cert for HTTPS).

But, when the actual Let's Encrypt servers try to connect to your site they get the DNS failures.

I just saw a SERVFAIL from the Let's Debug test (see results here). And, your problems were related to DNS also. I don't have a good guess as to what is wrong. Hopefully another volunteer will.

devahaminapurnama · October 10, 2022, 3:27am

Thankyou @MikeMcQ I also saw several other users having the same error as me, and I will wait for the DNS expert for this case and also Plesk Team Support will monitor this discussion for their improvement, for now they are using BIND as their DNS server.

rg305 · October 10, 2022, 5:38pm

Part of the problem might be that the two authoritative servers for the zone rafik.sch.id resolve to the same (single) IP:

Name:    up1.citrahost.com
Address: 103.123.16.140

Name:    up2.citrahost.com
Address: 103.123.16.140

All requests to both are being sent to one IP; That doubles the normal number of requests and might be triggering an IPS type system along the way.

In any case, having one single IP respond for a DNS zone is a very weak design and should be changed.

devahaminapurnama · October 11, 2022, 2:26am

Our plesk server more than year with same A record and NS without problem before, it's new policy from Let's Encrypt should have more than one IP to avoid single point of failure ?

rg305 · October 11, 2022, 3:58pm

No, it's just logical that a single point of failure will eventually fail.

devahaminapurnama · October 12, 2022, 1:20am

Okay, i will setup master-slave DNS server for plesk. Hope this help and I will confirm later.

devahaminapurnama · October 12, 2022, 1:05pm

[UPDATED 12/10/2022]

I just tried to create a new domain and install SSL, the main domain is in Cloudflare DNS and for this scenario I created 2 A records from the domain pointing to my two servers, without creating master-slave DNS as @rg305 said. I hope all ASN from my ISP are allowed to request Let's Encrypt installation where my company IP is in segment 103.123.16.0/22

In this plan I make 2 A records
plesk1.awankilat.my.id -> 103.123.16.140
plesk2.awankilat.my.id -> 103.123.16.178

For details, please see the screenshot.

Anyway, thankyou very much to the Community Leader @rg305 and @MikeMcQ for helping or anyone who read this case.

MikeMcQ · October 12, 2022, 1:13pm

Thanks for the update.

So, works using a different DNS provider (Cloudflare) rather than the CitraHost DNS.

You might try changing your other domain to use Cloudflare too.

devahaminapurnama · October 13, 2022, 1:15am

I don't know what happen to my server, but now both Citrahost and CloudFlare DNS can install Let's Encrypt SSL. This happened right after I opened the problem here, previously for the last 3 months I couldn't install Let's Encrypt SSL at all either using CloudFlare or Citrahost DNS.

devahaminapurnama · October 21, 2022, 3:45pm

Could not issue an SSL/TLS certificate for thkforum2022.com
Details

Could not issue a Let's Encrypt SSL/TLS certificate for thkforum2022.com. Authorization for the domain failed.

Details

Invalid response from https://acme-v02.api.letsencrypt.org/acme/authz-v3/167110499016.

Details:

Type: urn:ietf:params:acme:error:dns

Status: 400

Detail: DNS problem: query timed out looking up A for thkforum2022.com; DNS problem: query timed out looking up AAAA for thkforum2022.com

devahaminapurnama · October 21, 2022, 3:47pm

last week the other server is fine for issue SSL. but now encountered to error like this
IssueFromLetsEncrypt

ERROR

A test authorization for wplesk.citrahost.com to the Let's Encrypt staging service has revealed issues that may prevent any certificate for this domain being issued.

103.123.16.178: Fetching http://wplesk.citrahost.com/.well-known/acme-challenge/VLO1RvsRwcSjw5uyiyn3YuS-xlDNsQvYeWmBLoIh6YA: Timeout during connect (likely firewall problem)

IssueFromLetsEncrypt

ERROR

A test authorization for uplesk.citrahost.com to the Let's Encrypt staging service has revealed issues that may prevent any certificate for this domain being issued.

103.123.16.140: Fetching http://uplesk.citrahost.com/.well-known/acme-challenge/3fhNVUk7WhCl84Nm_iE71et94_C6gJxqTaPZRDYmKz4: Timeout during connect (likely firewall problem)

Port 80 and 443 is opened, all of my server is fine nothing changes on firewall side.

rg305 · October 21, 2022, 5:59pm

Those statements contradict each other.
Something has changed.
Something is now blocking the challenge requests.
Other than the "unchanged firewall", are there any other devices/programs/systems/etc. that might be able to block incoming HTTP requests from specific IPs/Networks/Lists/etc.?

devahaminapurnama · October 22, 2022, 1:51am

you can scan ports 80 and 443 to prove that ports really open

MikeMcQ · October 22, 2022, 2:30am

Yes, I see the ports open from my own test server. And, a Let's Debug test (here) does two test connections. It's first from its own test system succeeds but the second uses the Let's Encrypt staging system and that, again, fails with timeout.

This most likely is due to a firewall blocking the IP(s) that the LE servers use.

devahaminapurnama · October 22, 2022, 2:42am

hmm, but all of my firewall already turn off when issue Let's Encrypt sir

MikeMcQ · October 22, 2022, 2:59am

Are you able to view your nginx access log? You should see up to 4 challenge requests from Let's Encrypt in that log.

Try running a Let's Debug test and then check the logs. Show us what you see

devahaminapurnama · October 22, 2022, 3:25am

error.log
2022/10/22 06:39:29 [crit] 248002#0: *124293 SSL_do_handshake() failed (SSL: error:141CF06C:SSL routines:tls_parse_ctos_key_share:bad key share) while SSL handshaking, client: 198.199.89.210, server: 103.123.16.178:443
2022/10/22 07:40:50 [crit] 248002#0: *125415 SSL_do_handshake() failed (SSL: error:141CF06C:SSL routines:tls_parse_ctos_key_share:bad key share) while SSL handshaking, client: 178.62.253.99, server: 103.123.16.178:443
2022/10/22 08:44:09 [crit] 248002#0: *126955 SSL_do_handshake() failed (SSL: error:141CF06C:SSL routines:tls_parse_ctos_key_share:bad key share) while SSL handshaking, client: 178.62.105.178, server: 103.123.16.178:443

access.log

127.0.0.1 - - [22/Oct/2022:10:13:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
127.0.0.1 - - [22/Oct/2022:10:14:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
127.0.0.1 - - [22/Oct/2022:10:15:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
127.0.0.1 - - [22/Oct/2022:10:16:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
127.0.0.1 - - [22/Oct/2022:10:17:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
127.0.0.1 - - [22/Oct/2022:10:18:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
114.119.132.9 - - [22/Oct/2022:10:18:53 +0700] "GET /sitemapmobile900.xml HTTP/1.1" 301 162 "-" "Mozilla/5.0 (compatible;PetalBot;+https://webmaster.petalsearch.com/site/petalbot)"
127.0.0.1 - - [22/Oct/2022:10:19:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
157.230.26.103 - - [22/Oct/2022:10:19:48 +0700] "|\xB2\xA2v0v" 400 150 "-" "-"
157.230.26.103 - - [22/Oct/2022:10:19:49 +0700] "\x16\x03\x01\x00{\x01\x00\x00w\x03\x03\xEA\x83\xDC\xB9kP\xDC\xB54\xE5\xD2+d5\xD9n\x0E\x13(\xC3\x10\xB8\x12Y\x13\xD8\xAE\x8BA\xB6h\xE1\x00\x00\x1A\xC0/\xC0+\xC0\x11\xC0\x07\xC0\x13\xC0\x09\xC0\x14\xC0" 400 150 "-" "-"
157.230.26.103 - - [22/Oct/2022:10:19:49 +0700] "\x16\x03\x01\x00{\x01\x00\x00w\x03\x03\x85\x06\xC7\x10\xCC\xA3\xBC\xE6?\xF0(\x12\x95\xA0\xD0\xF9\xE2\xFD\x9A\xAA\x1CA\x08@M\x01\xC7:\xA0\x90\xFDr\x00\x00\x1A\xC0/\xC0+\xC0\x11\xC0\x07\xC0\x13\xC0\x09\xC0\x14\xC0" 400 150 "-" "-"
127.0.0.1 - - [22/Oct/2022:10:20:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
127.0.0.1 - - [22/Oct/2022:10:21:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
127.0.0.1 - - [22/Oct/2022:10:22:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
198.199.101.62 - - [22/Oct/2022:10:22:38 +0700] "GET /hudson HTTP/1.1" 404 196 "-" "Mozilla/5.0 zgrab/0.x"
127.0.0.1 - - [22/Oct/2022:10:23:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
127.0.0.1 - - [22/Oct/2022:10:24:21 +0700] "GET /nginx-status HTTP/1.1" 200 109 "-" "Python-urllib/3.6"
114.119.132.88 - - [22/Oct/2022:10:24:40 +0700] "GET /sitemapimages96.xml HTTP/1.1" 301 162 "-" "Mozilla/5.0 (compatible;PetalBot;+https://webmaster.petalsearch.com/site/petalbot)"

MikeMcQ · October 22, 2022, 3:31am

There are no entries related to Let's Debug or Let's Encrypt in either of those logs

Are you sure your DNS is pointing to the right system?

Does result of this command match your DNS IP?

curl -4 http://ifconfig.co

From a Let's Debug test, you should see a request for this that returns a 200 HTTP status code:

http://uplesk.citrahost.com/.well-known/acme-challenge/letsdebug-test

(you must have created a letsdebug-test file otherwise this would fail with 404 - which is more normal)

MikeMcQ · October 22, 2022, 3:38am

Oh, I just realized you are showing the logs for wplesk (I see it's IP in the error log).

I previously linked to a Let's Debug test for uplesk domain

Let me know which you are testing now

Still, you should see at least a result in your access log for the letsdebug-test request from a Let's Debug test. One for wplesk is here showing a 404 response (here) (you must not have created a letsdebug-test file on this server which is fine)