After update and renewal - server empty response (curl error 52)

gg4u · February 20, 2022, 9:30pm

Hello,
I can't find a solution.

About a month ago I had to renew certbot. Since then, subdomains occasionally send empty responses, and could not find a solution.

Each time I was apparently able to find a fix, like here:

The website was turning back, running again. But then after some time I check back the website:

curl discovery.nifty.works
curl: (52) Empty reply from server

And cannot find a solution.

I now tried again something else, I removed certbot and reinstalled via snapd:

I have certificates installed, included a wildcard one. And the main site works. But none of the subdomains.

Nginx configuration is fine and web servers up and running.

I set up DNS with _acme.challenge and it seems fine.

What's wrong ? I can't find a way to get back sites on subdomains, and find a solution that stays. At this point, I am even not sure that what I tried earlier (delete and new of new certificates; renewal of new certificates; different threads here) was a real solution to the problem.

Cannot find the error, but website won't appear, server sends no reply.


certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Found the following certs:
  Certificate Name: nifty.works-0001
    Serial Number: 499040cba88628d0aa87137afa0673395bf
    Key Type: RSA
    Domains: *.nifty.works
    Expiry Date: 2022-05-21 19:37:44+00:00 (VALID: 89 days)
    Certificate Path: /etc/letsencrypt/live/nifty.works-0001/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/nifty.works-0001/privkey.pem
  Certificate Name: nifty.works-0002
    Serial Number: 3297630963202b8da5e9d6cf8e88b7ff905
    Key Type: RSA
    Domains: nifty.works www.nifty.works
    Expiry Date: 2022-05-21 20:00:17+00:00 (VALID: 89 days)
    Certificate Path: /etc/letsencrypt/live/nifty.works-0002/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/nifty.works-0002/privkey.pem
  Certificate Name: nifty.works
    Serial Number: 3ac0fd9f46125d08d80d90819e173663457
    Key Type: RSA
    Domains: nifty.works
    Expiry Date: 2022-05-18 15:14:24+00:00 (VALID: 86 days)
    Certificate Path: /etc/letsencrypt/live/nifty.works/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/nifty.works/privkey.pem

certbot --version
certbot 1.23.0

dig _acme-challenge.nifty.works

; <<>> DiG 9.10.3-P4-Ubuntu <<>> _acme-challenge.nifty.works
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6699
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;_acme-challenge.nifty.works.	IN	A

;; AUTHORITY SECTION:
nifty.works.		233	IN	SOA	ns-935.awsdns-52.net. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400

;; Query time: 2 msec
;; SERVER: 50.116.61.5#53(50.116.61.5)
;; WHEN: Sun Feb 20 21:26:35 UTC 2022
;; MSG SIZE  rcvd: 140

nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Osiris · February 20, 2022, 9:34pm

Your nginx returns the same error over HTTP, so this is not an issue with TLS nor with the certificate.

gg4u · February 20, 2022, 9:56pm

@Osiris can you please explain ?

the last time certbot applied changes to the nginx configurations and sites were working fine.
Now nothing is displayed again.

But nginx -t shows OK configuration.

I also noted that:
the variable on _acme.challenge.nifty.works point to a XXXXXX.auth.acme-dns.io that cannot be solved.

I once tried to set certbot using this tutorial :

that made use of:

to generate the XXXXXX.auth.acme-dns.io

but now I also found that snap filled all my disk ?

df
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/root       24285116 23084556         0 100% /
devtmpfs         1002280        0   1002280   0% /dev
tmpfs            1006460        0   1006460   0% /dev/shm
tmpfs            1006460   107452    899008  11% /run
tmpfs               5120        0      5120   0% /run/lock
tmpfs            1006460        0   1006460   0% /sys/fs/cgroup
cgmfs                100        0       100   0% /run/cgmanager/fs
tmpfs             201292        0    201292   0% /run/user/1000
/dev/loop0        113536   113536         0 100% /snap/core/12725
/dev/loop1         63488    63488         0 100% /snap/core20/1328
/dev/loop2         44160    44160         0 100% /snap/certbot/1788
/dev/loop3         15360    15360         0 100% /snap/certbot-dns-route53/1370

(Why things are so complicated )

gg4u · February 20, 2022, 10:10pm

Darn.

Ok, it turned out disk was full for a massive log file was created and indeed the issue was not related with certbot.

However, please help me understand if the configuration is OK, because here and there the same problem of empty response from server appeared also on the main sites (the one on https) after having updated certbot, and I tried different things that occasionally made the site working again.

Indeed the variable on the DNS _acme.challenge.nifty.works seems to be incorrrect, but dig _acme.challenge.nifty.works gives no error. I now set the DNS as TXT as following the instructions on :

but on digitalocean guide was said to set CNAME:

Also, please instruct how to fix errors on dns using the version on snap now that I installed it (I don't know the difference with other versions on apt-get).

Thank you..

MikeMcQ · February 20, 2022, 10:56pm

The name should be _acme-challenge.nifty.works
Note the dash after _acme not a period

gg4u · February 21, 2022, 9:06am

Yes, you're right it was just a typo here.

_acme-challenge.nifty.works is named correctly, and dig _acme-challenge.nifty.works does not return error.

but the variable in it XXXX.auth.acme-dns.io cannot be reached.

If I browse https://www.nifty.works it seems everything ok.

But if I do

curl https://www.nifty.works
curl: (60) SSL certificate problem: Invalid certificate chain
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

Please advise how to fix and how to debug.

9peppe · February 21, 2022, 10:21am

Oh, but it does.

# dig txt  _acme-challenge.nifty.works

; <<>> DiG 9.16.1-Ubuntu <<>> txt _acme-challenge.nifty.works
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40537
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;_acme-challenge.nifty.works.   IN      TXT

;; ANSWER SECTION:
_acme-challenge.nifty.works. 300 IN     TXT     "2c483785-ae9b-446b-8fc6-4fe5c59c8f43.auth.acme-dns.io"

;; Query time: 15 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Mon Feb 21 11:20:10 CET 2022
;; MSG SIZE  rcvd: 122

You have to use a CNAME record, not a TXT one.

Osiris · February 21, 2022, 10:26am

Well, while incorrect it technically isn't returning an error

9peppe · February 21, 2022, 10:27am

I cannot reproduce this.

# curl -I https://www.nifty.works
HTTP/1.1 200 OK
Server: nginx/1.10.3 (Ubuntu)
Date: Mon, 21 Feb 2022 10:26:49 GMT
Content-Type: text/html
Content-Length: 25387
Last-Modified: Tue, 28 Jan 2020 21:02:35 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "5e30a16b-632b"
Accept-Ranges: bytes

Is ISRG Root X1 in your system trust store? (Or did you fix your chain by yourself?)

MikeMcQ · February 21, 2022, 3:05pm

@gg4u You probably have an out of date CA certificate store on that machine. Your server responds saying it is nginx 1.10.3 which is very old so I guess your Ubuntu and CA cert store might also be very old.

If you need help debugging your curl chain problem please run this:

curl -v  https://letsencrypt.org 2>&1 |grep CAfile

Then, show results of this command replacing CAfile with the name shown from above:

grep -Ei 'ISRG|DST|R3' CAfile | grep '#'

(example: grep -Ei 'ISRG|DST|R3' /etc/ssl/certs/ca-certificates.crt)

system · March 23, 2022, 3:06pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.