Problem renewing cert

I received a renewal warning email from the expiry bot today so ran

sudo certbot renew --dry-run

…and it’s failing with error:

  • The following errors were reported by the server:

    Domain: api-stage.example.com
    Type: tls
    Detail: remote error: tls: handshake failure

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A record(s) for that domain
    contain(s) the right IP address. Additionally, please check that
    you have an up-to-date TLS configuration that allows the server to
    communicate with the Certbot client.

  • The following errors were reported by the server:

    Domain: links-stage.anotherexample.com
    Type: tls
    Detail: remote error: tls: handshake failure

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A record(s) for that domain
    contain(s) the right IP address. Additionally, please check that
    you have an up-to-date TLS configuration that allows the server to
    communicate with the Certbot client.

How can I determine the issue and steps to fix?

Hi @fegoze,

It would be great to know your real domain name.

Are you sure that both of these hosts are running a web server (the same web server that they were running when you first obtained the certificate) that can speak HTTPS on port 443? Or, if you weren’t running a web server when you first obtained the certificate and you used --standalone, are you still not running a web server?

You don’t mention which OS your host is running. If it happens to be macOS, check out this topic for one possible cause & solution.

Hi @schoen,

I generated the certs using:
sudo certbot certonly --webroot -w [public app folder] -d [domain name]

On re-reading the error message I’ve realised that the second error is from an old certificate that isn’t used anymore.

There are two certs:
stage.example.com - dry-run renewal ok
api-stage.example.com - dry-run renewal error

The web servers are listening on different ports, nginx is setup to proxy 443 traffic to each based on server_name.

There is an additional rule for stage.example.com that redirects http to https, api-stage.example.com is only served over https.

Renewal process:

stop the applications
stop nginx
run the renewal command
start the applications
start nginx

Maybe there is a better way to do this?

In any case even when nginx and the applications are stopped, there is the handshake error.

Hi @hwine,

Thanks for the idea, OS is Ubuntu 16.04.2 x64.

I’m having the same issue. I’m using Cloudflare and I found a temporary fix is to pause the DNS and HTTP Proxy (CDN) (look at screenshot), run the certbot, and then turn it back on. I was hoping I could find a more permanent solution however.

Image: http://cloud.murfasa.com/1a1j2G3v1Q3f

Hi @murfasa, I’m also using cloudflare. I had to disable it during initial cert generation but I think read in another thread at the time that it wasn’t necessary to disable for renewals.

Same problem here. Renewal behind Cloudflare was not working. Both webroot and nginx methods were not working. I disabled CDN and it worked. My domain is 💩.brashear.me (💩.brashear.me).

I was getting a tls handshake error from certbot with CF proxy enabled.

I’m not sure if this information can solve everyone’s problems in this thread, but:

Using the --apache or --nginx methods (which use an authentication method called TLS-SNI-01) is completely incompatible with using Cloudflare or any other CDN or remote reverse proxy. That is true for both initial issuance and renewal. If you use these methods, you will always have to disable the CDN temporarily for renewals.

You can also be using --apache if you just ran certbot without telling it what method to use, because it is the default method if you have an Apache server.

If you used --apache, it will also be used automatically for renewals. Some people have had renewal failures because they weren’t behind a CDN when they first issued the certificate, but subsequently added a CDN after HTTPS was already working.

(If you plan to be behind Cloudflare permanently, you don’t necessarily need a Let’s Encrypt certificate at all; Cloudflare has its own CA partnership and can issue publicly trusted certificates for your site automatically. In this case, you can also use Cloudflare’s own free “origin certificates”, which are only trusted by Cloudflare but don’t require using a Let’s Encrypt client. The origin certificate protects the connection between Cloudflare and your origin server. As far as I know, there is no loss of security by using a Cloudflare origin certificate.)

Using the --webroot method (which uses an authentication method called HTTP-01) is compatible with CDNs, but you must have HTTP enabled on the CDN and it must actually be able to serve content from your origin server. (Redirections from HTTP to HTTPS either on the origin server or on the CDN, or both, are OK as long as a visitor ultimately gets the content when requesting a specific HTTP URI. Blanket redirections from specific HTTP URIs to the top level of the HTTPS site are not OK because the content the visitor gets doesn’t match.)

Finally, if you’re using --standalone, the default behavior is to use TLS-SNI-01. However, --standalone can also use HTTP-01 if you specific --preferred-challenges http-01.

I’m happy to deal with further questions, but I hope this summary is helpful to at least some people in this thread.

Can you elaborate on why is it incompatible? I’ve used LE behind cloudflare for a long time and never had issues until now. Is there something that may have changed?

Regarding not needing LE for Cloudflare, in my org users on our LAN go to the servers directly (no cloudflare), while external nets go through cloudflare. Being able to get a publicly-valid certificate enables this use case.

[quote="KeenRivals, post:10, topic:36738, full:true"]
Can you elaborate on why is it incompatible? I've used LE behind cloudflare for a long time and never had issues until now. Is there something that may have changed?[/quote]

Sure, the TLS-SNI-01 method makes a TLS connection to the host pointed to by the DNS record and sends an SNI extension in its ClientHello requesting to connect to a particular long random string plus .acme.invalid. It expects the TLS server to respond by serving a particular certificate. If the server does not do this, the TLS-SNI-01 challenge fails.

When you're behind Cloudflare, Cloudflare is terminating your TLS sessions for you and hence it doesn't know to respond to the .acme.invalid request with any particular certificate.

This is how TLS-SNI-01 has always worked, since the first day of Let's Encrypt's operation. If you were passing a challenge from behind a CDN, that CDN had special Let's Encrypt support of some kind, or it wasn't terminating TLS, or you were passing some other challenge (HTTP-01 or DNS-01). The HTTP-01 challenge generally works fine behind a CDN because it's about returning particular content at the HTTP level, and the CDN is willing to forward the HTTP request to the origin, which returns the appropriate content in response to a particular URI.

That makes sense!

1 Like

Thank you for the great explanation! I think it my case I was passing the HTTP-01 challenge, but after configuring the server to redirect HTTP to HTTPS, that somehow broke renews since they had to go through Cloudflare.

Hi @schoen, I updated the nginx config to redirect HTTP to HTTPS for api-stage.example.com.

I added a test file to [project dir]/public/.well-known. That’s where the challenge gets written isn’t it?

The test file contents is being returned when I navigate to
http://api-stage.example.com/.well-known/test.txt
in a in a browser.

The renewal still fails for api-stage.example.com but succeeds for stage.example.com.

Hi @schoen, I just noticed from the command (sudo certbot renew --dry-run) output that the challenges for each cert are different.

http-01 challenge for stage.example.com
...
tls-sni-01 challenge for api-stage.example.com

Where is the challenge type specified?

It is actually in /.well-known/acme-challenge, not just /.well-known.

The authenticator plugin that you selected when you first ran Certbot to obtain the certificate (or ran certonly a subsequent time to renew it) gets saved in the authenticator property in the file in /etc/letsencrypt/renewal corresponding to that particular certificate.

Different authenticator plugins use different challenge types. The apache plugin, which is suggested by default if you are running Apache and didn't specify any plugin, uses the TLS-SNI-01 challenge type, which requires a direct (not proxied or intermediated) TLS connection to your server on port 443, and hence doesn't work behind a CDN or reverse proxy or similar other device.

The webroot plugin, which is used if you specified --webroot or chose it from a menu, uses the HTTP-01 challenge type which involves creating those files in /.well-known/acme-challenge and accessing them with HTTP over port 80. This method does work behind a CDN or reverse proxy.

Hi @schoen,

Renewal configs:

api-stage.example.com:

authenticator = standalone

stage.example.com:

authenticator = webroot

What is the best way to update the certificate from standalone to webroot?

Run

certbot certonly --cert-name api-stage.example.com --force-renewal -a webroot -w /var/www/wherever/the/webroot/is

Hi @schoen,

I was able to renew the cert using the command, the renewal config got updated to authenticator = webroot, and certbot renew --dry-run now succeeds for both certs.

Thanks!

Great, I’m glad that helped!