Certbot-auto: Client lacks sufficient authorization

I know this question has been asked several times, but I need some specific help as I’ve been taking advice from several other threads and nothing has worked to this point.

I’m trying to renew my SSL certificate in Ubuntu 14.04 using certbot-auto renew, and I’m running an Apache2 server and nginx. I’m getting the following output for certbot-auto renew:

  root@PostgreSQLServer:/# sudo certbot-auto renew
  Saving debug log to /var/log/letsencrypt/letsencrypt.log

  -------------------------------------------------------------------------------
  Processing /etc/letsencrypt/renewal/my-domain.com.conf
  -------------------------------------------------------------------------------
  Cert is due for renewal, auto-renewing...
  Renewing an existing certificate
  Performing the following challenges:
  http-01 challenge for my-domain.com
  Waiting for verification...
  Cleaning up challenges
  Attempting to renew cert from /etc/letsencrypt/renewal/my-domain.com.conf produced an unexpected error: Failed authorization procedure. my-domain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://my-domain.com/.well-known/acme-challenge/ailNmgZADpb4QBipKM57sOi9w3PwNkwBwVFiRYs7i40: "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
  <html><head>
  <title>404 Not Found</title>
  </head><body>
  <h1>Not Found</h1>
  <p". Skipping.

  All renewal attempts failed. The following certs could not be renewed:
    /etc/letsencrypt/live/my-domain.com/fullchain.pem (failure)
  1 renew failure(s), 0 parse failure(s)

  IMPORTANT NOTES:
   - The following errors were reported by the server:

     Domain: my-domain.com
     Type:   unauthorized
     Detail: Invalid response from
     http://my-domain.com/.well-known/acme-challenge/ailNmgZADpb4QBipKM57sOi9w3PwNkwBwVFiRYs7i40:
     "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
     <html><head>
     <title>404 Not Found</title>
     </head><body>
     <h1>Not Found</h1>
     <p"

     To fix these errors, please make sure that your domain name was
     entered correctly and the DNS A record(s) for that domain
     contain(s) the right IP address.

I ensured the .well-known folder exists in /var/www/my-domain.com/public_html and when I navigate to http://my-domain.com/.well-known/ in my browser, I am able to see the contents of that directory. I also added an acme-challenge folder in .well-known and included a test.txt file for testing; I was able to access the directory and the text file in my browser.

I found that the acme-challenge folder was not created when I ran the certbot-auto command so it seems to be a permissions issue. I am running certbot-auto as root, but also gave write permissions to www-data user on the .well-known and acme-challenge folders (root and www-data users are running the apache2 and nginx processes).

Even after granting that write permission, I still get the 404 error detailed above.

I also have an automated cert renewal process running via crontab and I have the output being logged to a local file. In that log file, I see the renewal request seemed to be working correctly until certbot-auto was upgraded from 0.9.3 to 0.10.1. Here’s a sample from the log file when the upgrade took place:

  -------------------------------------------------------------------------------
  Processing /etc/letsencrypt/renewal/offensively-bad.com.conf
  -------------------------------------------------------------------------------

  The following certs are not due for renewal yet:
    /etc/letsencrypt/live/offensively-bad.com/fullchain.pem (skipped)
  No renewals were attempted.

  -------------------------------------------------------------------------------
  Processing /etc/letsencrypt/renewal/offensively-bad.com.conf
  -------------------------------------------------------------------------------

  The following certs are not due for renewal yet:
    /etc/letsencrypt/live/offensively-bad.com/fullchain.pem (skipped)
  No renewals were attempted.
  Upgrading certbot-auto 0.9.3 to 0.10.1...
  Replacing certbot-auto...
  Creating virtual environment...
  Installing Python packages...
  Installation succeeded.

  -------------------------------------------------------------------------------
  Processing /etc/letsencrypt/renewal/offensively-bad.com.conf
  -------------------------------------------------------------------------------

  All renewal attempts failed. The following certs could not be renewed:
    /etc/letsencrypt/live/offensively-bad.com/fullchain.pem (failure)
  IMPORTANT NOTES:
   - The following errors were reported by the server:

     Domain: offensively-bad.com
     Type:   unauthorized
     Detail: Invalid response from
     http://offensively-bad.com/.well-known/acme-challenge/tkSc8l-r1XVPIF5TosTbEXiYMa8sQnoXEjAEgAwRoqI:
     "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
     <html><head>
     <title>404 Not Found</title>
     </head><body>
     <h1>Not Found</h1>
     <p"

     To fix these errors, please make sure that your domain name was
     entered correctly and the DNS A record(s) for that domain
     contain(s) the right IP address.

You can see that after upgrading, the process began failing with a 404 message.

I’ve tried all the advice I could find online and am completely stumped, so any help would be much appreciated. Thanks in advance!

I think the primary thing here is to check /etc/letsencrypt/renewal/offensively-bad.com.conf for correctness of i’s contents, primarily the information beneath [[webroot_map]]. It should list something like:

[[webroot_map]]
offensively-bad.com = /path/to/what/certbot/thinks/is/the/correct/webroot/path/initially/specified/by/the/user

Could you check that to correspond to /var/www/offensively-bad.com/public_html/?

2 Likes

I spent a lot of time trying to figure out why I was also getting “Invalid response” for the acme challenge. My investigations led me to think that the auto renewal is placing the challenge files to a wrong place.

I had configured my nginx to serve the challenge files from /var/www/.well-known, but it always said 404 to the renew command even though I could serve my own files from that location. I then installed inotify-tools and added a watch for the directory inotifywait -m /var/www/.well-known/acme-challenge/ and saw that the renew command really doesn’t put anything there. I guess it places them somewhere else (probably to the place they were initially placed).

After that revelation I got mine renewed by issuing the command certbot renew --webroot-path /var/www/.

Hopefully there’s and advice for you here somewhere :slight_smile:

Thank you, @Osiris! This did the trick. Much appreciated!

1 Like

Well I did get a message indicating the certificate was renewed, but I’m still seeing that the cert is expired when I look at my website through some SSL checkers online, and my browser indicates the site is not secure.

Does it take time to renew the cert or is there something else I need to do? I’m a noob when it comes to SSL so apologies in advance if this is something simple I’m overlooking. Thanks for all the help.

Disregard this comment… I just had to restart nginx

You can use “certbot renew --post-hook 'service nginx reload'” to have Certbot automatically reload Nginx when it renews any certificates. :slight_smile:

1 Like

Thanks, @mnordhoff. I setup a cron job with this.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.