Renewal failed: Connection refused - status 400

Help
#1

Hello,

Before getting in the details, some years ago I had setup my let's encrypt certificates with dehydrated, so that's what I am still using.
Last week-end, my certificate renewing failed with a "Connection refused" error.
I was still using api v1, so I thought, good tiome to move over to v2. Unfornately, I still get failures when I renew.
I have multiple domains, but it fails at the first one already...

My domain is: bla.crepier.net

I ran this command: dehydrated -c -x

It produced this output:

ERROR: Challenge is invalid! (returned: invalid) (result: {
  "type": "http-01",
  "status": "invalid",
  "error": {
    "type": "urn:ietf:params:acme:error:connection",
    "detail": "During secondary validation: Fetching http://bla.crepier.net/.well-known/acme-challenge/dnvBF6j_gNAHBdYDG2ZeNuVw3qDMuRtmdPAYUBbcr4g: Connection refused",
    "status": 400
  },
  "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/8316167329/_1UqBw",
  "token": "dnvBF6j_gNAHBdYDG2ZeNuVw3qDMuRtmdPAYUBbcr4g",
  "validationRecord": [
    {
      "url": "http://bla.crepier.net/.well-known/acme-challenge/dnvBF6j_gNAHBdYDG2ZeNuVw3qDMuRtmdPAYUBbcr4g",
      "hostname": "bla.crepier.net",
      "port": "80",
      "addressesResolved": [
        "94.23.53.29"
      ],
      "addressUsed": "94.23.53.29"
    }
  ]
})

My web server is (include version): apache 2.4

The operating system my web server runs on is (include version): debian 9

I can login to a root shell on my machine (yes or no, or I don't know): yes

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 0.6.5

What I do not understand is that when inspecting the apache logs, I only see sucessful requests:

3.128.26.105 - - [02/Nov/2020:19:11:33 +0100] "GET /.well-known/acme-challenge/dnvBF6j_gNAHBdYDG2ZeNuVw3qDMuRtmdPAYUBbcr4g HTTP/1.1" 200 292 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
64.78.149.164 - - [02/Nov/2020:19:11:33 +0100] "GET /.well-known/acme-challenge/dnvBF6j_gNAHBdYDG2ZeNuVw3qDMuRtmdPAYUBbcr4g HTTP/1.1" 200 292 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"

I have fail2ban working as well, so I am now starting to wonder whether maybe it somehow banned one of the ips of let's encrypt.

Everything worked just fine for the past years.

So now, I am not sure where to look for the solution, but I have 30 days to get to it.
Anybody has a clue ?

2 Likes
#2

Let's Encrypt uses multiple vantage points to validate a hostname. That's what is meant by "secondary validation" in the error message. See for more information:

So you're seeing only two of the four (one primary and three secondary) validation attempts. This means 50 % of the validation attempts are blocked somehow.

3 Likes
#3

Sadly that only says "Connection refused" not "Connection refused from IP w.x.y.z"
But, yes:

1 Like
#4

Problem solved after stopping on fail2ban jail. I have new certificates!
Two ips used for validation were caught some time ago it seems like.

Yet it would be handy if the ips failing would be provided so that it could be unbanned more easily for instance.

3 Likes
#5

That's not going to happen though. The IP's can change at any notice and I'm assuming some security by obscurity might be welcome to limit the BGP attack vector.

2 Likes
#6

Providing the failed IP in the message doesn't shed any more light on the IPs being used than the IPs themselves being used.

Any "BGP attacker" (with half a brain) could get the list of current IPs in less than one minute.
Yes those IPs could (and should be able to) change without notice.
This isn't about providing, and maintaining, an up-to-date list of testing IPs.
This is about not divulging an IP to those that have failed its' test.
That seems to be harming more than it is helping (IMHO).
They are a "key witness" and are not stepping forward to testify.

2 Likes