Timeout during connect (likely firewall problem)

Help
1

Hello!
I have several websites on my server.
LE certificates have been renewing successfully for a long time.
No firewall settings have been changed for a long time.
And no server settings have been changed for a long time.
Yesterday, certificates for several websites were successfully renewed

Today, when trying to renew a certificate for another site, I received the following error message:

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: xxxxxx.yyy
  Type:   connection
  Detail: During secondary validation: XXX.XXX.XXX.XXX: Fetching http://xxxxxx/.well-known/acme-challenge/YYYYYYYYYYYYYYYYYYYY: Timeout during connect (likely firewall problem)

and a similar message for the www-domain for xxxxxx.yyy

In the access.log, when attempting to renew the LE certificate, there are six entries for successful access (return code 200, size 87 bytes) to the .well-known/acme-challenge/YYYYYYYYYYYYYYYYYYYYY from the US and Sweden.

What is LE missing?
How can I find the cause of the problem?

2

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:

I ran this command:

It produced this output:

My web server is (include version):

The operating system my web server runs on is (include version):

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know):

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

3 Likes
3

Do you LE staff member?
For what this list of questions for total noobs?
The problem is that LE started using some unusual locations for verification. Not Europe or the US.
I would know which.

4

When you opened this thread in the Help section, you should have been provided with a questionnaire. Maybe you didn't get it somehow (which is weird), or you've decided to delete it (and make our life a lot harder). In any case, all the answers to this questionnaire are required. N00b or not.

While the locations of the secondary validation servers aren't a secret, they are also not published. I would have expected also some requests from Asia. In total there should be 4 to 5 requests per challenge.

As you're seeing this error with the "During secondary validation" mention, some aren't able to connect. Usually this is due to geoblocking at the side of the webserver.

2 Likes
5

How many of those 6 had the same challenge token? Because at most you would see 5 challenges for a successful challenge with current layout. Since your request failed with "Secondary" at least 2 of the 4 secondary centers failed to reach you (b/c currently 1 failure is tolerated).

My guess is your "six" were for two different domain names and only 3 succeeded. Which is not enough.

And, since you said you knew of US and Europe LE locations you probably have been blocking the Asian location for a long time. Which would still have allowed your requests to succeed since one secondary is allowed to fail. But, doesn't tolerate even a transient failure for any of the other 3 secondary centers. The LE validation locations haven't changed in nearly 2 years.

4 Likes
6

There was 3 successfull challenges for each ot two acme tokens.
First token: 2 "US" and 1 "Sweden" successfull challenges.
Second token: 2 "US" and 1 "Sweden" successfull challenges.
Nothing else in the log.

In which country is the Asian LE verification center located?

7

Then you are missing one of the US locations and the Asian.

See: Public IP addresses of Issuance process - #4 by mcpherrinm

Pay particular attention to the notice that these locations may change at any time and may change without any notice. LE has talked about adding more locations and possibly adding different service providers. There is no automated way to know the LE validation centers.

There are other options than having to be aware of and dependent on specific LE infrastructure.

This wiki explains multi-perspective validation. The section I linked gives other methods and suggestions to give you more reliability: Multi-Perspective Validation & Geoblocking FAQ One option is to switch to using a DNS Challenge. Then LE only needs access to your DNS servers which are commonly accessible world-wide anyway.

5 Likes
8

Singapore - ok!
Renew successfull, thank you!

9

Are you now seeing 5 challenge URLs in your access log?

Or still just 4 and missing one of the US secondaries?

4 Likes
10

Are you sure there are three validations from the US? From Matthews post I deduced only at a minimum 2 from the US: at minimum 1 primary and at minimum 1 secondary. I cannot deduce from the post that there would be at least more than 1 secondaries from the US.

3 Likes
11

Last I checked about a year ago there were two US secondaries (AWS, one on each coast). I now use DNS Challenges so doesn't matter to me personally. I don't know that I'd have time today to run a test.

Matthew's post only lists 3 countries for secondary centers but we know there were 5 total so one was duplicated.

Do you use http or tls-alpn challenges? The locations would show up in your logs

4 Likes
12

Doublechecked, your info is still correct :slight_smile: 1x US primary, 2x US secondary, 1x Sweden and 1x Singapore.

But as you also noticed, this could change at any notice :wink:

4 Likes
13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.