[SOLVED] Renewing server failed to connect

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: sheffieldboulder.uk / kimura.no-ip.info

I ran this command:

certbot renew --agree-tos

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/kimura.no-ip.info.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for kimura.no-ip.info
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (kimura.no-ip.info) from /etc/letsencrypt/renewal/kimura.no-ip.info.conf produced an unexpected error: Failed authorization procedure. kimura.no-ip.info (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://kimura.no-ip.info/.well-known/acme-challenge/_w_lEsEQ4Fmbt6dMGzGBu8NvTvkGW4RbkKOAOUOAmiA: "   <!DOCTYPE html>
<html lang="en" id="document" dir="ltr">
<head prefix="og: http://ogp.me/ns# article: http://ogp.me/ns/article# ". Skipping.

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/sheffieldboulder.uk.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for sheffieldboulder.uk
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (sheffieldboulder.uk) from     /etc/letsencrypt/renewal/sheffieldboulder.uk.conf produced an unexpected error: Failed authorization procedure. sheffieldboulder.uk (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://sheffieldboulder.uk/.well-known/acme-challenge/mDu4ZJ1TzDC42TDuaZt9iFueFKGCi9lvRS6QkoEK448: "<!DOCTYPE html>
<html lang="en" dir="ltr" class="no-js">
<head>
    <meta charset="utf-8" />
    <title>well-known_acme-challeng". Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/kimura.no-ip.info/fullchain.pem (failure)
  /etc/letsencrypt/live/sheffieldboulder.uk/fullchain.pem (failure)

-------------------------------------------------------------------------------

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/kimura.no-ip.info/fullchain.pem (failure)
  /etc/letsencrypt/live/sheffieldboulder.uk/fullchain.pem (failure)
-------------------------------------------------------------------------------
2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: kimura.no-ip.info
   Type:   unauthorized
   Detail: Invalid response from
   http://kimura.no-ip.info/.well-known/acme-challenge/_w_lEsEQ4Fmbt6dMGzGBu8NvTvkGW4RbkKOAOUOAmiA:
   "<!DOCTYPE html>
   <html lang="en" id="document" dir="ltr">
   <head prefix="og: http://ogp.me/ns# article:
   http://ogp.me/ns/article# "

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.
 - The following errors were reported by the server:

   Domain: sheffieldboulder.uk
   Type:   unauthorized
   Detail: Invalid response from
   http://sheffieldboulder.uk/.well-known/acme-challenge/mDu4ZJ1TzDC42TDuaZt9iFueFKGCi9lvRS6QkoEK448:
   "<!DOCTYPE html>
   <html lang="en" dir="ltr" class="no-js">
   <head>
       <meta charset="utf-8" />
       <title>well-known_acme-challeng"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

My web server is (include version): nginx version: nginx/1.12.2

The operating system my web server runs on is (include version): ArchLinux (no version, rolling release)

My hosting provider, if applicable, is: OVH

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

Initially I had a different error but I solved that after reading that I need to disable redirects of http/port 80 which I have in place and have done so in the /etc/nginx/nginx.conf and restarted the service to ensure redirects didn’t happen. Now hit this problem.

At some point I’ve screwed up the certificate for sheffieldboulder.uk too as I now can’t access that site due to a certificate error.

If it makes any difference when I first obtained certificates I setup nginx with root /var/lib/letsencrypt/ for both sites and then changed the root to root /usr/share/nginx/html/sheffieldboulder/ and root /usr/share/nginx/html/kimura/. Realising that the necessary ~/.well-known/acme-challenge directories were not present in either of these I created them, ensuring permissions were those for the web server user…

mkdir -p /usr/share/nginx/html/sheffieldboulder/.well-known/acme-challenge /usr/share/nginx/html/kimura/.well-known/acme-challenge
chown -R http:http /usr/share/nginx/html/sheffieldboulder/.well-known /usr/share/nginx/html/kimura/.well-known

…but that didn’t help. I noticed that certbot is reporting that its looking for http://localhost/ but am unsure why (a guess would be it recognises its on the target domain).

Wondering if I’d be better off just creating new certificates rather than refreshing (currently one is due to expire 2018-01-19 and I expect to get a renewal notice for the other in the not too distant future).

This is all about url to filesystem mapping.

To get around this, just create the directory tree /.well-known/acme-challenge/ on both domains and put a file test.txt there. As long as you cannot reach this file when viewing http://domainname/.well-known/acme-challenge/test.txt with a web browser, the directories are created at a wrong location.

Besides: creating new certificates is the same as renewing existing ones. But you should definitely renew instead of creating a new set of certificates to avoid confusion.

Thanks for the prompt reply.

I create the suggested test.txt…

echo 'Hello World!' >> /usr/share/nginx/html/kimura/.well-known/acme-challenge/test.txt
echo 'Hello World!' >> /usr/share/nginx/html/sheffieldboulder/.well-known/acme-challenge/test.txt
chown -R http:http /usr/share/nginx/html/sheffieldboulder/.well-known/acme-challenge /usr/share/nginx/html/kimura/.well-known/acme-challenge/

…and they are now viewable kimura.no-ip.info/.well-known/acme-challenge/text.txt and sheffieldboulder.uk/.well-known/acme-challenge/text.txt.

Unfortunately I’m now locked out for an hour :rofl:

# certbot renew --agree-tos
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/kimura.no-ip.info.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Attempting to renew cert (kimura.no-ip.info) from /etc/letsencrypt/renewal/kimura.no-ip.info.conf produced an unexpected error: urn:acme:error:rateLimited :: There were too many requests of a given type :: Error creating new authz :: Too many failed authorizations recently.. Skipping.

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/sheffieldboulder.uk.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Attempting to renew cert (sheffieldboulder.uk) from /etc/letsencrypt/renewal/sheffieldboulder.uk.conf produced an unexpected error: urn:acme:error:rateLimited :: There were too many requests of a given type :: Error creating new authz :: Too many failed authorizations recently.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/kimura.no-ip.info/fullchain.pem (failure)
  /etc/letsencrypt/live/sheffieldboulder.uk/fullchain.pem (failure)

-------------------------------------------------------------------------------

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/kimura.no-ip.info/fullchain.pem (failure)
  /etc/letsencrypt/live/sheffieldboulder.uk/fullchain.pem (failure)
-------------------------------------------------------------------------------
2 renew failure(s), 0 parse failure(s)

Will try again later.

Tried again this morning and still no joy, I can reach the ~/.well-known-acme-challenge/test.txt on each server (see here and here), but no joy…

26 certbot renew --agree-tos
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/kimura.no-ip.info.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for kimura.no-ip.info
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (kimura.no-ip.info) from /etc/letsencrypt/renewal/kimura.no-ip.info.conf produced an unexpected error: Failed authorization procedure. kimura.no-ip.info (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://kimura.no-ip.info/.well-known/acme-challenge/hIRrRyo_u8EbRHzp3qeGRLXr1BK-XDIUqdjklazXEPg: "<!DOCTYPE html>
<html lang="en" id="document" dir="ltr">
<head prefix="og: http://ogp.me/ns# article: http://ogp.me/ns/article# ". Skipping.

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/sheffieldboulder.uk.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for sheffieldboulder.uk
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (sheffieldboulder.uk) from /etc/letsencrypt/renewal/sheffieldboulder.uk.conf produced an unexpected error: Failed authorization procedure. sheffieldboulder.uk (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://sheffieldboulder.uk/.well-known/acme-challenge/lyIWF847xTyTJnGDZ2fZ8UlGiS6S-z64WsIBOcaaTmc: "<!DOCTYPE html>
<html lang="en" dir="ltr" class="no-js">
<head>
    <meta charset="utf-8" />
    <title>well-known_acme-challeng". Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/kimura.no-ip.info/fullchain.pem (failure)
  /etc/letsencrypt/live/sheffieldboulder.uk/fullchain.pem (failure)

-------------------------------------------------------------------------------

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/kimura.no-ip.info/fullchain.pem (failure)
  /etc/letsencrypt/live/sheffieldboulder.uk/fullchain.pem (failure)
-------------------------------------------------------------------------------
2 renew failure(s), 0 parse failure(s)

I notice though that certbot is trying to connect on http and yet redirects are being made to https. I thought I had (temporarily) disabled redirects as I’ve uncommented the following settings for listen on port 80 and commented out those for port 443 and have definitely restarted nginx to ensure these are used but the redirect still happens. Here are the lines from my server config (/etc/nginx/nginx.conf)

# kimura.no-ip.info                                                                                                                                                                                                                                                                                                      
server {
    listen 80;
    listen [::]:80;
    # listen 443 ssl http2;                                                                                                                                                                                                                                                                                              
    # listen [::]:443 ssl http2;                                                                                                                                                                                                                                                                                         
    server_name  kimura.no-ip.info;
    root   /usr/share/nginx/html/kimura;
...
}
server {
    listen 80;
    listen [::]:80;
    # listen 443 ssl http2;                                                                                                                                                                                                                                                                                              
    # listen [::]:443 ssl http2;                                                                                                                                                                                                                                                                                         
    server_name  sheffieldboulder.uk;
    root   /usr/share/nginx/html/sheffieldboulder;
...
}

Not sure where I’ve gone wrong this time?

Can you show us the contents of this file? It looks to me like Certbot is not configured to save the challenge file to the same place where /.well-known/acme-challenge/ is actually served from.

Have a look at another approach: instead of retrieving test.txt, try a non-existing filename:

$ wget -qO - http://kimura.no-ip.info/.well-known/acme-challenge/testd | head -10
<!DOCTYPE html>
<html lang="en" id="document" dir="ltr">
<head prefix="og: http://ogp.me/ns# article: http://ogp.me/ns/article# fb: http://ogp.me/ns/fb# place: http://ogp.me/ns/place# book: http://ogp.me/ns/book#">
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>
well-known_acme-challenge_testd    [kimura.no-ip.info]
</title>
<meta name="generator" content="DokuWiki"/>

So you see: a non-existing path does not create a 404, but gets a (PHP generated) Dokuwiki page. You should disable the routing to Dokuwiki’s index.php for any path that starts with /.well-known/acme-challenge/.

And, like @_az states: check if certbot actually places the files at the correct location.

Thanks both for prompt replies…

cat /etc/letsencrypt/renewal/kimura.no-ip.info.conf
# renew_before_expiry = 30 days
version = 0.19.0
archive_dir = /etc/letsencrypt/archive/kimura.no-ip.info
cert = /etc/letsencrypt/live/kimura.no-ip.info/cert.pem
privkey = /etc/letsencrypt/live/kimura.no-ip.info/privkey.pem
chain = /etc/letsencrypt/live/kimura.no-ip.info/chain.pem
fullchain = /etc/letsencrypt/live/kimura.no-ip.info/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = d7c279f685c1b33615230da68b1c39cd
authenticator = webroot
installer = None
webroot_path = /var/lib/letsencrypt,
[[webroot_map]]
kimura.no-ip.info = /var/lib/letsencrypt

The full server definition looks like this…

# kimura.no-ip.info                                                              
server {
    listen 80;
    listen [::]:80;
    # listen 443 ssl http2;
    # listen [::]:443 ssl http2;
    server_name  kimura.no-ip.info;
    root   /usr/share/nginx/html/kimura;
    index doku.php;
    #Remember to comment the below out when you're installing DokuWiki, and uncomment it when you're done.
    location ~ /(data/|conf/|bin/|inc/|install.php) { deny all; } # secure Dokuwiki
    # LestEncrypt : Certificates
    ssl_certificate /etc/letsencrypt/live/kimura.no-ip.info/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/kimura.no-ip.info/privkey.pem;
    ssl_trusted_certificate /etc/letsencrypt/live/kimura.no-ip.info/chain.pem;
location ~^/\.ht { deny all; } # also secure the Apache .htaccess files      
location @dokuwiki {
        #rewrites "doku.php/" out of the URLs if you set the userewrite setting to .htaccess in dokuwiki config
        rewrite ^/_media/(.*) /lib/exe/fetch.php?media=$1 last;
        rewrite ^/_detail/(.*) /lib/exe/detail.php?media=$1 last;
        rewrite ^/_export/([^/]+)/(.*) /doku.php?do=export_$1&id=$2 last;
        rewrite ^/(.*) /doku.php?id=$1&$args last;
    }
    location / { try_files $uri $uri/ @dokuwiki; }
location ~ \.php$ {
        try_files $uri =404;
        fastcgi_pass unix:/run/php-fpm/php-fpm.sock;
        fastcgi_index index.php;
        include fastcgi.conf;
    }
}

I’m far from a whiz with nginx and a quick search suggests that perhaps adding an explicit location for ~/.well-known/acme-challenge/ might do the trick, so something like…

location acme-challenge {
    root /usr/share/nginx/html/kimura/.well-known/acme-challenge/
}

…but I’m not 100% sure that is correct. Another thread seems to suggest I need to use try_files but I’m unsure what files certbot might be creating/using.

This is the problem. It is dropping the challenge in there rather than your actual nginx webroot.

You need to either:

  1. Change nginx to serve files from /var/lib/letsencrypt for the /.well-known/acme-challenge/ URL prefix, or
  2. Change the Certbot config's webroot paths from /var/lib/letsencrypt to /usr/share/nginx/html/kimura/ (or whatever the document root of the domain being edited is).

e.g. (1)

location /.well-known/acme-challenge/ {
    root /var/lib/letsencrypt;
}
2 Likes

Thank you _az I went with your suggestion...

...and have successfully renewed both certificates....

# certbot renew --agree-tos
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/kimura.no-ip.info.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for kimura.no-ip.info
Waiting for verification...
Cleaning up challenges

-------------------------------------------------------------------------------
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/kimura.no-ip.info/fullchain.pem
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/sheffieldboulder.uk.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for sheffieldboulder.uk
Waiting for verification...
Cleaning up challenges

-------------------------------------------------------------------------------
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/sheffieldboulder.uk/fullchain.pem
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/kimura.no-ip.info/fullchain.pem (success)
  /etc/letsencrypt/live/sheffieldboulder.uk/fullchain.pem (success)
-------------------------------------------------------------------------------

Both of your help has been very much appreciated, thank you.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.