Certbot failed to authenticate: connect

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: kznlegislature2021womenparliament.veedo.live

I ran this command:

sudo /snap/bin/certbot certonly --text --non-interactive --cert-name kznlegislature2021womenparliament.veedo.live --keep-until-expiring --renew-with-new-domains --renew-by-default --agree-tos --email craig@lateral.co.za --cert-path /etc/letsencrypt/live/kznlegislature2021womenparliament.veedo.live --standalone -d kznlegislature2021womenparliament.veedo.live --test-cert -v

It produced this output:

sudo /snap/bin/certbot certonly --text --non-interactive --cert-name kznlegislature2021womenparliament.veedo.live --keep-until-expiring --renew-with-new-domains --renew-by-default --agree-tos --email craig@lateral.co.za --cert-path /etc/letsencrypt/live/kznlegislature2021womenparliament.veedo.live --standalone -d kznlegislature2021womenparliament.veedo.live --test-cert -v
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Account registered.
Requesting a certificate for kznlegislature2021womenparliament.veedo.live
Performing the following challenges:
http-01 challenge for kznlegislature2021womenparliament.veedo.live
Waiting for verification...
Challenge failed for domain kznlegislature2021womenparliament.veedo.live
http-01 challenge for kznlegislature2021womenparliament.veedo.live

Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
Domain: kznlegislature2021womenparliament.veedo.live
Type: connection
Detail: Fetching http://kznlegislature2021womenparliament.veedo.live/.well-known/acme-challenge/BmPVFgs-jCb880xnN9ABXB8rn7zrpJkChghM_stWl5Q: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet.

Cleaning up challenges
Some challenges have failed.

My web server is (include version): not running - checked with ps and with curl

The operating system my web server runs on is (include version):
Ubuntu 18.04.5 LTS

My hosting provider, if applicable, is:
microsoft azure

I can login to a root shell on my machine (yes or no, or I don't know):
yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.18.0

BUT:

  1. while certbot is running, I can run a curl from my computer or from another cloud machine, and I get:
    curl http://kznlegislature2021womenparliament.veedo.live
    ACME client standalone challenge solver%

  2. On the second run, it worked. But then I've got a test certificate. So I thought, ok, yay, just a DNS thing, so I reran for a proper cert (without --test-cert) and it failed again. I don't want to do this too often with failure in case I get banned (I've tried about 4 times already).

My guess is you've got a couple of certificate servers, and one or more of them haven't updated their DNS recently, or something like that (although this is a new domain I've just configured, so it should not be in DNS yet). I've no other ideas... do you?

Best, C

2

Why do you think the issue might be due to DNS? Did you change the IP address in your DNS settings recently?

Further more, I find the options for certbot you're using quite elaborate and very non-standard. Did you pick them up from sort of (misguided) guide perhaps? Almost all options used are, in my opinion, not useful.

1 Like
3

Thank you for the response!

The command is run out of an installation script. I've just created a VM, configured the DNS server, and am then running the certbot command, so I want no interaction. Which arguments are not-required? I specify the cert-name and path because sometimes, if the files already exist, certbot uses different paths, and --standalone, --agree-to, --email, --text, --non-interactive and -d get me there. The --renew-* and --keep-until-expiring are from a previous script I was using, and are more useful if I'm doing a renewal, but I want the script to manage renewal as well, which it does with those flags.

When the script didn't work I ran it by hand, to see what was wrong, hence the -v and the --test-cert.

Which options do you think could be causing the certbot servers to fail to connect to my machine?

1 Like
4

I've tried again with:

sudo /snap/bin/certbot certonly --non-interactive --agree-tos -m craig@lateral.co.za -d kznlegislature2021womenparliament.veedo.live --standalone --test-cert

Same problem persists:

  Domain: kznlegislature2021womenparliament.veedo.live
  Type:   connection
  Detail: Fetching http://kznlegislature2021womenparliament.veedo.live/.well-known/acme-challenge/18PGCeQbVWdE4AA-Mn4LbmNjLEKcxant_A-Im_xvcM4: Timeout during connect (likely firewall problem)
5

Now on a third or fourth attempt, it has succeeded:

Certificate is saved at: /etc/letsencrypt/live/kznlegislature2021womenparliament.veedo.live/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/kznlegislature2021womenparliament.veedo.live/privkey.pem

But this is a test certificate, so I'm back to where i was at the beginning: I need a valid certificate, and I cannot make too many bad requests of the certbot server before I get banned!

6

The option --renew-by-default, which is an alias for --force-renewal is a very dangerous option: it will overwrite perfectly fine certificates with a newly issued certificate, possibly running into rate limits or at least adding unnecessary load on the Let's Encrypt servers. This option is terribly named, I believe the original creator of it has much regret about it. It does NOT mean "please renew my certificate if it's time to renew", but "renew NOW, even if the cert isn't close to expiry yet!".

The option --keep-until-expiring is the exact opposite of the option above and is fine (makes sense actually) in a non-interactive situation.

The --text option doesn't have any function any longer since the removal of the curses interface in v0.10.0. I'm pretty sure you're not running a version of certbot older than that? I'm curious, how did you find out about that option anyway? It has been suppressed since that time too, so it isn't mentioned any longer in the documentation at all.

--cert-name should be fine for that, no need to specify the --cert-path.

Not sure, is it a single server behind a single IP address? No changes recently? Firewall in place?

1 Like
7

Yes, these options have been many years in the making, and I have carried them over from my scripts with the earliest versions of letsencrypt! I'm super glad to learn what --renew-by-default does. In fact, because of that option, I have implemented my own caching system which caches my certificates and doesn't call certbot unless I don't have the certs, exactly because certbot kept overriding my good certs. So glad to learn that was just because of a flag I was passing, and I can retire my caching system :slight_smile:
But I still have the problem, even with my newer flag set that I've just used above :frowning:

8

Yes, it's a single server, single IP address. As I say, the IP / DNS has just been setup. The IP comes from a reserved set we cycle through, so most probably has been used for other servers recently. There's no firewall on ports 80 or 443.

9

If it's a DNS issue, i.e.: the LE servers aren't using the correct IP addres, this should be visible in the detailed output of certbot: the authorization output should show you the IP address it tried to connect to. If it's the correct IP address, it's not a DNS issue. If it is an IP address which is not the current IP address of the server, it might be a DNS issue.

Speaking of DNS issues, please see kznlegislature2021womenparliament.veedo.live | DNSViz That tool isn't very happy about the nameservers saying the hostname dfree.datafree5.co does not exist. Which is odd, as the hostname 3110.dfree.datafree5.co actually does exist. Although I don't think this is related to your current issue. As Unboundtest shows here the resolver used by Let's Encrypt, Unbound, can resolve your hostname just fine.

1 Like
10

I get this in the letsencrypt.log file:

"error": {
        "type": "urn:ietf:params:acme:error:connection",
        "detail": "Fetching http://kznlegislature2021womenparliament.veedo.live/.well-known/acme-challenge/PbJ8Jr0hx4AXR4lE3BebAEy2A0Utz6FKtHeOOAceDYQ: Timeout during connect (likely firewall problem)",
        "status": 400
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/19554722440/BFNmBQ",
      "token": "PbJ8Jr0hx4AXR4lE3BebAEy2A0Utz6FKtHeOOAceDYQ",
      "validationRecord": [
        {
          "url": "http://kznlegislature2021womenparliament.veedo.live/.well-known/acme-challenge/PbJ8Jr0hx4AXR4lE3BebAEy2A0Utz6FKtHeOOAceDYQ",
          "hostname": "kznlegislature2021womenparliament.veedo.live",
          "port": "80",
          "addressesResolved": [
            "13.244.165.110"
          ],
          "addressUsed": "13.244.165.110"
        }
      ],

So it looks like the servers are using the correct IP, but not resolving.
Is there any reason that it works sometimes?
When it worked - a few minutes before - I got:

  "challenges": [
    {
      "type": "http-01",
      "status": "valid",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/210414938/iumiyw",
      "token": "d9C1D4Dwg7FizfhM6VxqHaE2oMXiTb5plsR0zm4J_tQ",
      "validationRecord": [
        {
          "url": "http://kznlegislature2021womenparliament.veedo.live/.well-known/acme-challenge/d9C1D4Dwg7FizfhM6VxqHaE2oMXiTb5plsR0zm4J_tQ",
          "hostname": "kznlegislature2021womenparliament.veedo.live",
          "port": "80",
          "addressesResolved": [
            "13.244.165.110"
          ],
          "addressUsed": "13.244.165.110"
        }
      ],
      "validated": "2021-08-06T18:17:04Z"
    }
  ]

Same IP So this doesn't look like a DNS issue. Some sort of networking glitch perhaps?

11

Now I've just got this error:

  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:dns",
        "detail": "No valid IP addresses found for kznlegislature2021womenparliament.veedo.live",
        "status": 400
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/19556156700/aINNiA",
      "token": "Vs_th_yccf9sIAb2cXyZlAwUsYGHswMXha1Tn31vn6M",
      "validated": "2021-08-06T19:50:07Z"
    }
  ]

Not sure what is going on! Everything resolving/connecting fine on my side, though...

12

What do you mean by "resolving"? Usually, the verb "to resolve" is used as synomym for the DNS process of "looking up a certain IP address of a hostname". And as you say, the IP address used by Let's Encrypt is correct, so the DNS resolving works fine.

Let's Encrypt uses multiple vantage points to validate from, but if the secondary vantage points don't work, it will show up in the specific error message. So it seems it's the main server intermittently succeeding and failing, which is quite weird..

1 Like
13

Sorry, what I mean was resolving the DNS to the correct IP, but not connecting.

14

Well.. Unless we see other people opening threads about strange intermitting connection time outs, I'm enclined to say if this is a network issue, it probably isn't an issue with the Let's Encrypt network specifically, if you're the only one having issues. Maybe it's a network problem closer to your location, but it would be rather hard to detect.. Especially as Let's Encrypt uses multiple end points to connect from.. I'm not sure how to proceed from now on I'm afraid..

1 Like
15

No worries, I have to call it a day anyway. We've had this issue a few times before, and it's usually been ok the following day, so I'll go to bed and hope it comes right.
It's a valid point about others not reporting the issue, I'll see what happens.
Thank you very much for the assistance and input.

2 Likes
16

@craigmj, do you use any software like Fail2ban or anything that would rate limit (or block) specific IPs?

17

Thank you for the response. It's a good idea - in general I do - but I've just checked and my scripts haven't yet installed f2b - that comes after I've got certs. I also checked iptables -L, and nothing is banned.
I'm on the Microsoft Azure South Africa North network, so I guess there's a chance that that network is blocking the incoming traffic from one or more of LetsEncrypts servers, but I don't know how to test for that unless I can find the originating IP in the logs somewhere?

1 Like
18

Here's some more detail. I see this error in /var/log/letsencrypt.log:

      "error": {
        "type": "urn:ietf:params:acme:error:connection",
        "detail": "Fetching http://kznlegislature2021womenparliament.veedo.live/.well-known/acme-challenge/yg5F4a2z0h6OudFh2eeV8LdikEWEHyzPnps5-nQ8NCQ: Timeout during connect (likely firewall problem)",
        "status": 400
      },

But earlier in the log file, I see:

2021-08-07 04:16:39,615:DEBUG:acme.standalone:::ffff:13.244.165.110 - - Incoming request
2021-08-07 04:16:39,615:DEBUG:acme.standalone:::ffff:13.244.165.110 - - Serving HTTP01 with token 'yg5F4a2z0h6OudFh2eeV8LdikEWEHyzPnps5-nQ8NCQ'
2021-08-07 04:16:39,615:DEBUG:acme.standalone:::ffff:13.244.165.110 - - "GET /.well-known/acme-challenge/yg5F4a2z0h6OudFh2eeV8LdikEWEHyzPnps5-nQ8NCQ HTTP/1.1" 200 -
2021-08-07 04:16:39,662:DEBUG:acme.standalone:::ffff:13.244.165.110 - - Incoming request
2021-08-07 04:16:39,662:DEBUG:acme.standalone:::ffff:13.244.165.110 - - Serving HTTP01 with token 'yg5F4a2z0h6OudFh2eeV8LdikEWEHyzPnps5-nQ8NCQ'
2021-08-07 04:16:39,662:DEBUG:acme.standalone:::ffff:13.244.165.110 - - "GET /.well-known/acme-challenge/g5F4a2z0h6OudFh2eeV8LdikEWEHyzPnps5-nQ8NCQ HTTP/1.1" 200 -
2021-08-07 04:16:39,715:DEBUG:acme.standalone:::ffff:13.244.165.110 - - Incoming request
2021-08-07 04:16:39,715:DEBUG:acme.standalone:::ffff:13.244.165.110 - - Serving HTTP01 with token 'yg5F4a2z0h6OudFh2eeV8LdikEWEHyzPnps5-nQ8NCQ'
2021-08-07 04:16:39,715:DEBUG:acme.standalone:::ffff:13.244.165.110 - - "GET /.well-known/acme-challenge/yg5F4a2z0h6OudFh2eeV8LdikEWEHyzPnps5-nQ8NCQ HTTP/1.1" 200 -
2021-08-07 04:16:40,093:DEBUG:acme.standalone:::ffff:13.244.165.110 - - Incoming request

Surely that means that the standalong webserver has served the request?

The IP of the request is the 'kznlegislature2021womenparliament' IP, which is the 'router/firewall' for the actual box doing the webserving. But we've used this structure and configuration extensively and without issues. Hmm... I'm going to try a different IP configuration and see whether that helps...

1 Like
19

I realised this server was configured from a 'new' range of firewalls. I switched to one from our 'previous' range, and everything works perfectly.
In part-conclusion, this is clearly an issue with the network of the particular firewall machine. I will update once I've debugged that. Thank you all for the details help.

2 Likes
20

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.