Timeout on certification creation behind IPv6 and IPv4 ADSL box

Help
1

Hi,

I was able to create the SSL certificat in manual mode three months ago.
I moved the server to another location keeping the domain name but with different IP address without any problem.
After three month, I tried to renew the certificat without success.
So, I revoked and deleted the certificate that I manually created using certbot manual option.
Now, I tried to create the certificat directly from the NAS hosting the server web but all my attempts with webroot and standelone plugins have failed with the same message: Timeout.

My current understanding of the issue is that Let’s Encrypt tries to connect my server for the challenge but it is not able to get the challenge file. I think Let’s Encrypt uses the IPv6 IP address of the box without trying the IPv4.
But the web server is not reachable from the IPv6 address. Let’s Encrypt should use the IPv4 address.

Did I something wrong or there is a problem with Let’s Encrypt?

Thank you in advance for your help.
Best Regards.

My domain is: zebulon.freeboxos.fr

With the standelone plugin, I ran this command:
/etc/init.d/nginx stop && certbot certonly --standalone --preferred-challenges http --rsa-key-size 4096 -w /var/www -d zebulon.freeboxos.fr

With the webroot plugin, I ran this command:
certbot certonly --webroot --rsa-key-size 4096 -w /var/www -d zebulon.freeboxos.fr

It produced this output:
Failed authorization procedure. zebulon.freeboxos.fr (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://zebulon.freeboxos.fr/.well-known/acme-challenge/DehoMuQzHy4sw-7Ln4IU3NxZ5-LPa-wFC6fdRWv7XF0: Timeout

IMPORTANT NOTES:

From the log
"challenges": [
{
“type”: “http-01”,
“status”: “invalid”,
“error”: {
“type”: “urn:acme:error:connection”,
“detail”: “Fetching http://zebulon.freeboxos.fr/.well-known/acme-challenge/DehoMuQzHy4sw-7Ln4IU3NxZ5-LPa-wFC6fdRWv7XF0: Timeout”,
“status”: 400
},
“uri”: “https://acme-v01.api.letsencrypt.org/acme/challenge/GSYUR4u-vK-g_Wg_YfeKxj2ZnXi3Gd72_VICQn8KDCU/1545277360”,
“token”: “DehoMuQzHy4sw-7Ln4IU3NxZ5-LPa-wFC6fdRWv7XF0”,
“keyAuthorization”: “DehoMuQzHy4sw-7Ln4IU3NxZ5-LPa-wFC6fdRWv7XF0.rUMxsjRcBj_YtGCCb5TnTl7TXO0XTpeMUKrE36HRkHc”,
“validationRecord”: [
{
“url”: “http://zebulon.freeboxos.fr/.well-known/acme-challenge/DehoMuQzHy4sw-7Ln4IU3NxZ5-LPa-wFC6fdRWv7XF0”,
“hostname”: “zebulon.freeboxos.fr”,
“port”: “80”,
“addressesResolved”: [
“88.171.241.65”,
“2a01:e35:8abf:1410::1”
],
“addressUsed”: “2a01:e35:8abf:1410::1”,
“addressesTried”: []
}
]
},

My web server is (include version): Nginx 1.10.3

The operating system my web server runs on is (include version): Debian 8 Jessie

My hosting provider, if applicable, is: Free with Freebox v6
I think my problem come form here. But the first time I was also behind a Freebox v6 working in IPv6 and IPv4.

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

2

Hi @Antipseudo,

If your site is not available via IPv6, you should stop advertising the IPv6 address in DNS. Then the certificate issuance can proceed. As you may have noticed, Let's Encrypt is trying to validate in IPv6—because your DNS record is telling us that it's OK to do so!

This issue has been discussed quite a bit on this forum.

https://community.letsencrypt.org/search?q=ipv6%20fallback

3

Hi shoen,

Thank you for you answer.
I take some time to read the posts on this forum but I’m not able to know if their problems are related to mine.
All of them are about IPv6 but most of them are linked to a problem of configuration.
There is a request to change how Let’s Encrypt behave in this post Prefer IPv4 for validation when ACME client requests are IPv4. But I’m not sure that the proposal is correct.
The "Ipv6 address returns 404, ipv4 is good, but failed to verify the domain " (Ipv6 address returns 404, ipv4 is good, but failed to verify the domain) topic sound closer to my issue. But I do not understand the conclusion. Moreover, it seems that in this use case the user is able to configure the network has he want but this is not my case.

First of all, I would like clarify one point. I cannot force the box to use only IPv4 and I cannot change the DNS configuration of the access provider (owner of the box). The DNS returns two IP addresses (one IPv4 and another IPv6) because the network support both versions.

All the applications in Internet behave fine with my DNS configuration. As far as know, only Let’s Encrypt has a problem with that. For example I’m able to reach the server from my web browser and ssllabs.com (https://www.ssllabs.com/ssltest/) is able to get the certificates of the server.

My understanding of the current IPv6 specification is that the Internet applications have to prefer IPv6 but try IPv4 if IPv6 failed. This situation is mainly due to the fact that we have to live with an Internet mixing IPv4 and IPv6 computers.
This is exactly my situation. The box support IPv6 and IPv4. The box itself can be reach in IPv6 but the network behind is IPv4.

So according to the requested service, it is possible to use IPv4 or IPv6. In my case HTTP (80) and HTTPS (443) use IPv4 but the box itself is able to do some other stuff in IPv6.

Best Regards,
Vincent.

4

You need to get your IPv6 configured correctly. If you advertise AAAA records, make sure all public services are working. Let’s encrypt is not the problem.

5

Hi Vincent

Your DNS records seem to be a bit problematic

Andrei

6

Hi,

Thank you ahaw021.

I’m not an expert on the dig and host command but both are able to resolve the domain giving the IPv4 address.
antipseudo@MONPC:~$ dig zebulon.freeboxos.fr +nocomments +noquestion +noauthority +noadditional +nostats

; <<>> DiG 9.10.3-P4-Ubuntu <<>> zebulon.freeboxos.fr +nocomments +noquestion +noauthority +noadditional +nostats
;; global options: +cmd
zebulon.freeboxos.fr. 1655 IN A 88.171.241.65
antipseudo@MONPC:~$ host zebulon.freeboxos.fr
zebulon.freeboxos.fr has address 88.171.241.65
zebulon.freeboxos.fr has IPv6 address 2a01:e35:8abf:1410::1

I don’t know if there is something bad in the DNS configuration of my access provider, the only thing that I know is that I do not have any issue with my web browser or online tools checking the configuration of my server (except the fact that my SSL certificate are expired now).

Let’s Encrypt was able to create the certificate but it seems that something has been changed in the its behaviour and now Let’s Encrypt is no more able to renew the certificates.

If I well understood the Yomamma’s answer, it seems that Let’s Encrypt does not allow to use the IPv4 address if the DNS return an IPv6 address. But internet is not necessary used for HTTP or HTTPS only so we can have some services running in IPv6 and some others in IPv4.

So, is it a problem in the DNS configuration or Let’s Encrypt does not check the IPv4 address if the DNS returns also an IPv6 address?

Best Regards,
Vincent.

7

Currently, Let's Encrypt checks only the IPv6 address if it is present via an AAAA record. This behavior changed a couple of months ago, so it might have been different when you originally obtained your certificate.

8

To clarify: Let's Encrypt will check the IPv4 address after the IPv6 address if it fails for TLS-SNI-01 challenges. This will also happen for HTTP-01 challenges but not until after the next release on Thursday due to an outstanding bug.

1 Like
9

Hi,

Thank you schoen and cpu for your answers.
So, if I well understood the behaviour of Let’s Encrypt will be changed again to check the challenge on the IPv4 if it fails in HTTP or HTTPS for the IPv6.
Is there a ticket or bug/feature tracking item that I can use to know when this modification will be supported?

Best Regards,
Vincent.

1 Like
10

The bug has already been closed and the fix is merged to master in Boulder. The only remaining thing is deploying it to production.

This typically happens on the Thursday of every week, so I would expect the problem to be resolved late afternoon pacific time on Thursday the 20th. We announce deploys with a changelog on the status page and that's the best place to watch to know when this will be supported.

Hope that helps!

11

Great ! :slight_smile:
Thank you very much for your help and support.
I really appreciate the promptness of our responses.

I’m not ready today to open IPv6 behind the box. The box does not have any firewall to secure the incoming and outgoing traffics between Internet and all the network elements connected to the box (WiFi). With IPv4, I can use the NAT to restrict the incoming traffic to a specific list of network elements and services (and easily block the P2P traffic of guest equipments). in France, we are automatically responsible of the illegal actions (like illegal download) done through our own Internet connection.

I don’t know how many peoples encountered the same problem. But the fact that the Let’s Encrypt support the challenge on IPv4 and IPv6 allow me to reuse the domain name and DNS of the access provider for my personal network without compromising the security of my network by enabling IPv6. So, by enabling one option in the box and one simple certbot command line, I’m able to secure the communications between Internet and my personal network. This is just perfect !

Thank you Certbot and Let’s Encrypt :slight_smile:

Best Regards,
Vincent.

12

Salut Atipseudo,

J’ai exactement le même problème que toi sur mon NAS Synology : je ne peux plus renouveler mon certificat depuis la dernière expiration.
Je possède également le nom de domaine en freeboxos.fr.
Et j’ai le même log avec les 2 adresses IP, l’une en V4 et l’autre en v6, et Let’s Encrypt n’arrive pas à joindre le NAS sur la V6.
Je vois que la correction du bug déclaré aurait dû être déployée, on en sait plus ?

–

Hi Antipseudo,
I’ve exactly the same problem as yours. My domain is *.freeboxos.fr. 2 IP adresses are associated, one v4 and one v6 by the ISP.
The patch (https://github.com/letsencrypt/boulder/issues/2770) has not been release in production ?

13

Hello,

As Antipseudo and Skuair, I also encounter the same issue. My certificate has expired August, 4th and I identified in the log that the failing of the certificate renewal has begun in the beginning of July Month.

Thank your for your help.

14

@mgam, please create a separate thread for your issue, complete with some additional descriptions of how you’re attempting renewal, and any specific responses or logs that would help in troubleshooting. If you were having the exact same issue, the responses above should help you resolve it. If they did not, then you’re having a different issue.

15

Hi @jared.m

If I answered it is to confirm that other people having same issue. In any case, it seems the same problem. Indeed, I should have shared additional informations, here they are :

in the beggining of the logs i found that :
2017-07-06T04:58:01+02:00 NAS builtin-syno-letsencrypt-syno-letsencrypt: autorenew: syno-letsencrypt.cpp:350 Failed to renew /usr/syno/etc/certificate/_archive/hygpOy. {“error”:200,“file”:“client.cpp”,“msg”:“new-cert: Unexpect httpcode. (new-cert)”}
2017-07-14T16:24:06+02:00 NAS builtin-syno-letsencrypt-syno-letsencrypt: autorenew: syno-letsencrypt.cpp:350 Failed to renew /usr/syno/etc/certificate/_archive/hygpOy. {“error”:200,“file”:“client.cpp”,“msg”:“new-cert: Unexpect httpcode. (new-cert)”}

here, when I realize :
2017-08-03T15:57:03+02:00 NAS synoscgi_SYNO.Core.Certificate.LetsEncrypt_1_create[17165]: certificate.cpp:957 syno-letsencrypt failed. 200 [new_authz: unexpect httpcode.] 2017-08-03T16:10:25+02:00 NAS synoscgi_SYNO.Core.Certificate.LetsEncrypt_1_create[25108]: certificate.cpp:957 syno-letsencrypt failed. 200 [new-cert: Unexpect httpcode. (new-cert)] 2017-08-03T16:17:00+02:00 NAS synoscgi_SYNO.Core.Certificate.LetsEncrypt_1_create[29402]: certificate.cpp:957 syno-letsencrypt failed. 200 [new-cert: Unexpect httpcode. (new-cert)]

And here my last try:
2017-08-07T16:17:54+02:00 NAS synoscgi_SYNO.Core.Certificate.LetsEncrypt_1_create[5582]: certificate.cpp:957 syno-letsencrypt failed. 200 [new_authz: unexpect httpcode.]

I hope it would help.

Thank you

UP: That configuration has been set since the beginning of let’s encrypt and it always worked; the certificate did always automatically renew and I had nothing to do.

16

This is definitely a different issue. It may have the same symptom (you can’t get a certificate renewed), but there are numerous reasons. This really should be a new topic (they try to keep the forum to one topic per issue), but I suspect a moderator will split it for you.

What command is being run to effectuate this process? It looks like your server is producing some weird responses.

17

I don’t really know what commands are run. I’m using the wizard supplied by synology.

What makes me think it’s the same issue:

1- it is the domain name we (Skuair and Antipseudo) are using and the ISP. We have the same ISP and it provides a sub domain for each customer as sub.freeboxos.fr. Even if Antipseudo is a little be different.
2- With another domain (synology.me), I don’t have any problem from synology NAS to create a certificate with the same wizard
3- Moreover, the renewal of my box (CPE) worked well on 8 July 2017 (for the last renewal) and also using LE.

But anyway if it needs to split the topic, there is no problem for me. :slight_smile:

18

Hi,

Sorry for my late answer.
I just tried to create the certificate based on webroot plugin and It seems that Let’s Encrypt still not able to resolve the IPv4 address.

The command line is the following :

certbot certonly --webroot --rsa-key-size 4096 -w /var/www -d zebulon.freeboxos.fr

Please find below an extract of the certbot’ log:

Connection: keep-alive

{
  "identifier": {
    "type": "dns",
    "value": "zebulon.freeboxos.fr"
  },
  "status": "invalid",
  "expires": "2017-08-14T17:47:27Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:acme:error:connection",
        "detail": "Fetching http://zebulon.freeboxos.fr/.well-known/acme-challenge/dwJLXXEr3zjtIGBRVq7OiuEcWuj77i1Fu3twANhzVmU: Timeout",
        "status": 400
      },
      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/kGQdB9pa9FbfYNRkuRZYGEduUhQ5uApOwBeNNVpPNZs/1703330132",
      "token": "dwJLXXEr3zjtIGBRVq7OiuEcWuj77i1Fu3twANhzVmU",
      "keyAuthorization": "dwJLXXEr3zjtIGBRVq7OiuEcWuj77i1Fu3twANhzVmU.rUMxsjRcBj_YtGCCb5TnTl7TXO0XTpeMUKrE36HRkHc",
      "validationRecord": [
        {
          "url": "http://zebulon.freeboxos.fr/.well-known/acme-challenge/dwJLXXEr3zjtIGBRVq7OiuEcWuj77i1Fu3twANhzVmU",
          "hostname": "zebulon.freeboxos.fr",
          "port": "80",
          "addressesResolved": [
            "88.171.241.65",
            "2a01:e35:8abf:1410::1"
          ],
          "addressUsed": "2a01:e35:8abf:1410::1",
          "addressesTried": []
        }
      ]
    },
    {
      "type": "tls-sni-01",
      "status": "pending",
      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/kGQdB9pa9FbfYNRkuRZYGEduUhQ5uApOwBeNNVpPNZs/1703330134",
      "token": "eLlbewok8Lukrmc9YBMMxoJMykzyxAvhuXMabs77Pko"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/kGQdB9pa9FbfYNRkuRZYGEduUhQ5uApOwBeNNVpPNZs/1703330136",
      "token": "8g2ZHiT_8PNHoYCgGWjKdogrCiP405Wu1sFFscbnzSs"
    }
  ],
  "combinations": [
    [
      2
    ],
    [
      1
    ],
    [
      0
    ]
  ]
}

Certbot displays the following error message :

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: zebulon.freeboxos.fr
   Type:   connection
   Detail: Fetching
   http://zebulon.freeboxos.fr/.well-known/acme-challenge/dwJLXXEr3zjtIGBRVq7OiuEcWuj77i1Fu3twANhzVmU:
   Timeout

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.
19

In principle this is supposed to fall back to IPv4 according to the new behavior and we can ask @cpu why this is apparently not happening.

Nonetheless, you are still advertising an IPv6 address for your domain name that is unreachable. This is still a broken configuration, and Let’s Encrypt’s fallback behavior—whether it works properly or not!—is only a workaround for this misconfiguration. If you don’t want people to connect to your site via IPv6, you should remove the AAAA record from DNS.

20

Likely an instance of this case. We've reached a point of diminishing returns for implementing fallback workarounds so as summarized by @jsha's comment this is a won't fix for the time being and should be addressed by the website operator fixing their IPv6 configuration or removing the incorrect AAAA record.