The client lacks sufficient authorization


#1

Hello

I have a peculiar problem: On my server (Ubuntu 14.04 LTS) I have several virtual domains with a certificate from Let’s Encrypt.
I have installed the certbot cronjob for automatic renewal.
All was working since yet I think. In the weekly emails from cron I never found an error.

When I got this email today from LE, that TLS-SNI-01 validation is no longer supported, I just checked with a dry-run if there is any of this old domain validation used.
In this dry-run I noticed that only one domain shows an error:

#certbot renew --dry-run ... Processing /etc/letsencrypt/renewal/7fw.de.conf - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Cert not due for renewal, but simulating renewal for dry run Plugins selected: Authenticator apache, Installer apache Renewing an existing certificate Performing the following challenges: http-01 challenge for 7fw.de http-01 challenge for www.7fw.de Waiting for verification... Cleaning up challenges Attempting to renew cert (7fw.de) from /etc/letsencrypt/renewal/7fw.de.conf produced an unexpected error: Failed authorization procedure. 7fw.de (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://7fw.de/.well-known/acme-challenge/Elmxm6vVAAN93xRh4FW-palA_3utqc-C27QelSFkS10: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p", www.7fw.de (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.7fw.de/.well-known/acme-challenge/Ncl60Drtud7Rss2FAOOpjinJCPExHHCb7i7KJi6ueME: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p". Skipping. ...

I found several posts here with this issue, but I couldn’t resolve it.
The interesting thing is, that I have no .well-known directory in none of my virtual domains.
Neither in the one with the error, nor in any other without any error.

I checked the permissions and all is normal (www-data:www-data).
What could that be?
It is:
certbot --version
0.28

Thanks
frank


#2

Hi @franc,

You’ve found a bug in Certbot which we’re continuing to investigate. If you’d like to share your complete Apache configuration, it might help us figure out where the problem is.

A workaround for some people can be to switch to --webroot instead of --apache for this domain.


#3

7fw.de.conf is:

<VirtualHost *:80>
        ServerName 7fw.de
         ServerAlias www.7fw.de        
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/7fw.de
        <Directory /var/www/7fw.de>
                Options -Indexes +FollowSymLinks +MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>
        ErrorLog /var/log/apache2/error_7fw.log
        LogLevel warn
        CustomLog /var/log/apache2/access_7fw.log combined
        ServerSignature On

        DirectoryIndex index.php index.html

</VirtualHost>
<IfModule mod_ssl.c>

        <VirtualHost *:443>
                ServerName 7fw.de:443
                ServerAdmin webmaster@localhost
                ServerAlias www.7fw.de:443
                DocumentRoot /var/www/7fw.de
                ErrorLog /var/log/apache2/error_7fw.log
                LogLevel warn
                CustomLog /var/log/apache2/access_7fw.log combined
                <IfModule mod_rewrite.c>
                        RewriteEngine On
                        RewriteCond %{REQUEST_METHOD} ^TRACE
                        RewriteRule .* - [F]
                </IfModule>

                SSLEngine on

                SSLProtocol all -SSLv2
                SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM


                <Directory /var/www/7fw.de>
                        SSLRequireSSL
                        Options -Indexes -Includes -ExecCGI
                </Directory>

                ErrorDocument 400 /error_docs/bad_request.html
                ErrorDocument 401 /error_docs/unauthorized.html
                ErrorDocument  403 /error_docs/forbidden.html
                ErrorDocument 404 /error_docs/not_found.html
                ErrorDocument 500 /error_docs/internal_server_error.html
                ErrorDocument 405 /error_docs/method_not_allowed.html
                ErrorDocument 406 /error_docs/not_acceptable.html
                ErrorDocument 407 /error_docs/proxy_authentication_required.html
                ErrorDocument 412 /error_docs/precondition_failed.html

                ErrorDocument 415 /error_docs/unsupported_media_type.html
                ErrorDocument 501 /error_docs/not_implemented.html
                ErrorDocument 502 /error_docs/bad_gateway.html
                
                DirectoryIndex index.php index.html

                ServerAlias 7fw.de
                Include /etc/letsencrypt/options-ssl-apache.conf
                SSLCertificateFile /etc/letsencrypt/live/7fw.de/cert.pem
                SSLCertificateKeyFile /etc/letsencrypt/live/7fw.de/privkey.pem
                SSLCertificateChainFile /etc/letsencrypt/live/7fw.de/chain.pem
        </VirtualHost>

</IfModule>

#4

Hi @franc

a port number in the ServerName?

ServerName 7fw.de

is the regular version.


#6

Please show:
grep -Eri '7fw.de|servern|servera|virtualh|listen|rewrite|return|root' /etc/apache2


#7

I had the same issue (Ubuntu 14.04). I had 3 different certificates/domains with very similar setups, and one was failing with “Failed authorization procedure” after following the instructions linked from the TLS-SNI-01 email:

I just tried your workaround using the --webroot version, and it worked:
certbot certonly --webroot -w /var/www/foobar/public_html -d domain.com -d www.domain.com

I tried a sudo certbot renew --dry-run again after this, and the error no longer occurs.


#8

Hi @franc !

To help Certbot development: if in any way possible, could you re-run Certbot with --debug-challenges - command line parameter? This pauses Certbot right after the challenge configuration changes have been deployed, and waits for user to press any button to continue.

I would love to see optionally your complete Apache configuration at that point, if you are not comfortable in sending the whole configuration over, could you send the --debug-challenges version of the 7fw.de.conf.

Also a related question: does the 7fw.de webroot directory /var/www/7fw.de have an .htaccess file? If so, getting that would help us tremendously.

If you wish to do so, you can send the configuration files directly to me via email: joona.hoikkala@eff.org


#10

Nothing wrong with that, see ServerName:
ServerName voll-qualifizierter-Domainname[:port]


#11

It is allowed but not the recommended/normal.
It can lead to a false sense of configuration.
How does apache handle:

<VirtualHost FQDN1:PORT1>
     ServerName FQDN2:PORT2
     ServerAlias FQDN3:PORT3

How should that be handled?


#12

This gives me the same, if I select 1 in:

1: Apache Web Server plugin - Beta (apache)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)

If I select webroot there is no error, this works. I guess, the 1. is the default.
Is there something wrong with my Apache Web Server plugin?

I tried:

mkdir /var/www/7fw.de/public_html
certbot certonly --webroot -w /var/www/7fw.de/public_html -d 7fw.de -d www.7fw.de

and choose to renew the certificate, but I still have the same error.

I have the same thing (ServerName voll-qualifizierter-Domainname:port) in other conf files which run in certbot without error.


#13

It may be defined / possible. But it’s a source of potential problems.

In the last months, I’ve never seen such a configuration in this forum.

So it makes things easier if you remove that.


#14

The “default” is found in the last renewal conf:

This seems not up-to-date:

Yes, It is allowed but is all to easy to change one port and forget to change the other and apache will still run but won’t do what you think it should be doing.


#15

But how can I update that (Apache Web Server plugin - Beta)?
And still, this canont be the error cause, the other domains are working.

I took out the ports of the conf file for ServerName and ServerAlias but no change after that in the error message.


#16

I would follow instructions that match the O/S and web server from here: https://certbot.eff.org/all-instructions/


#17

That is exactly how I installed it in the first place, about half year ago.


#18

What irritates me is, that in:
/etc/letsencrypt/renewal/7fw.de.conf
(and all the other working domains) the certbot version (?) is mentioned:
version = 0.26.1
But when I command, I get:

#certbot --version
certbot 0.28.0

#19

The renewal file just contains the last version of Certbot used to renew the certificate. If it hasn’t been renewed recently, it might not be the current version of Certbot you have now. It doesn’t necessarily indicate a problem.


#20

That was silly.
I did the correct command now:
certbot certonly --webroot -w /var/www/7fw.de -d 7fw.de -d www.7fw.de
(without that “public_html” dir, which was just an example)
with " Renew & replace the cert" and got no errors. Now in my
/etc/letsencrypt/renewal/7fw.de.conf
I can read:

# renew_before_expiry = 30 days
version = 0.28.0
archive_dir = /etc/letsencrypt/archive/7fw.de
cert = /etc/letsencrypt/live/7fw.de/cert.pem
privkey = /etc/letsencrypt/live/7fw.de/privkey.pem
chain = /etc/letsencrypt/live/7fw.de/chain.pem
fullchain = /etc/letsencrypt/live/7fw.de/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = webroot
account = 25b80607468c31cc0402cd5f29d47ea0
server = https://acme-v02.api.letsencrypt.org/directory
webroot_path = /var/www/7fw.de,
[[webroot_map]]
7fw.de = /var/www/7fw.de
www.7fw.de = /var/www/7fw.de

Furthermore I had still errors in a dry-run of certbot renewal, because I had a second config-file for 7fw.de, the file:
/etc/letsencrypt/renewal/www.7fw.de.conf
which is from my beginnings with LE, when I knew nothing (and not as now only very, very little) about it. I think I thought I had to create another certificate for www.7fw.de, not only for 7fw.de.
So I just deleted it and now the error in the dry-run is gone.

I still dont know why I cannot use authenticator = apache for this domain, but need to use authenticator = webroot but as long as it works, I won’t complain :wink:

Thank you everybody for the help!

Btw: I am still a bit clumsy with formatting Markup in this “Discourse” Forum, especially in my first post (which I cannot edit anymore) sorry about it :blush:


#21

That was one of my first suspicions, but no, there is none.


#22

If it would not be too much trouble, I would still like for us to review the output of: