Renew Cert Failed - The client lacks sufficient authorization :: Invalid response

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: nextcloud.oxigen.sg

I ran this command: certbot --apache

It produced this output:
Failed authorization procedure. nextcloud.oxigen.sg (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://nextcloud.oxigen.sg/.well-known/acme-challenge/37r2DukHWVTqepqqLXHteKracqLRG4kGlLn2B2DWaG8 [116.202.30.75]: “\n\n404 Not Found\n\n

Not Found

\n<p”

IMPORTANT NOTES:

/.well-known/acme-challenge folder is not present. I can create this folder and a browser can see a test file in this folder.
DNS is correct.

ufw status - port 80 and 443 opened

I have another domain collabora.oxigen.sg on the same IP and the cert updated automatically.

My web server is (include version): Apache 2.4.38

The operating system my web server runs on is (include version): Debian 10

My hosting provider, if applicable, is: Hetzner

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): 0.31.0

2

Show us your apache config. (```pre before and ``` after, each on its own line)

You most probably will have to use the webroot plugin instead of the apache one.

3

Sorry, don’t quite understand this " ( ````pre` before and ````` after, each on its own line)"

# The ServerName directive sets the request scheme, hostname and port th at # the server uses to identify itself. This is used when creating # redirection URLs. In the context of virtual hosts, the ServerName # specifies what hostname must appear in the request's Host: header to # match this virtual host. For the default virtual host (this file) this # value is not decisive as it is used as a last resort host regardless. # However, you must set it for any further virtual host explicitly. ServerName nextcloud.oxigen.sg 
    ServerAdmin chris.chan@runbox.com
    DocumentRoot /var/www/nextcloud

<Directory /var/www/nextcloud/>

AllowOverride All

    <IfModule mod_headers.c>
            Header always set Strict-Transport-Security "max-age=15552000; i

ncludeSubDomains; preload"

<Directory /var/www/nextcloud/>
AllowOverride All

    # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
    # error, crit, alert, emerg.
    # It is also possible to configure the loglevel for particular
    # modules, e.g.
    #LogLevel info ssl:warn

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    # For most configuration files from conf-available/, which are
    # enabled or disabled at a global level, it is possible to
    # include a line for only one particular virtual host. For example the
    # following line enables the CGI configuration for this host only
    # after it has been globally disabled with "a2disconf".
    #Include conf-available/serve-cgi-bin.conf

SSLCertificateFile /etc/letsencrypt/live/nextcloud.oxigen.sg/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/nextcloud.oxigen.sg/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf

4 
```pre

paste your file here

```
5 

<IfModule mod_ssl.c>
<VirtualHost *:443>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        ServerName nextcloud.oxigen.sg

        ServerAdmin chris.chan@runbox.com
        DocumentRoot /var/www/nextcloud

#       <Directory /var/www/nextcloud/>
#               AllowOverride All
#       </Directory>

        <IfModule mod_headers.c>
                Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains; preload"
        </IfModule>

        <Directory /var/www/nextcloud/>
                AllowOverride All
        </Directory>

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
       #Include conf-available/serve-cgi-bin.conf

SSLCertificateFile /etc/letsencrypt/live/nextcloud.oxigen.sg/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/nextcloud.oxigen.sg/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>
6

I need to see the port 80 virtualhost too.

7 

<VirtualHost *:80>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        ServerName nextcloud.oxigen.sg

        ServerAdmin chris.chan@runbox.com
        DocumentRoot /var/www/nextcloud

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
RewriteEngine on
RewriteCond %{SERVER_NAME} =nextcloud.oxigen.sg
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
8

Try this command:

certbot -a webroot -w /var/www/nextcloud -i apache -d nextcloud.oxigen.sg
9 
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer apache
Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for nextcloud.oxigen.sg
Using the webroot path /var/www/nextcloud for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. nextcloud.oxigen.sg (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://nextcloud.oxigen.sg/.well-known/acme-challenge/Pm1_52BfpjoXrdKUEe4STSd4B8FpB9_bACu969qyBQ8 [116.202.30.75]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: nextcloud.oxigen.sg
   Type:   unauthorized
   Detail: Invalid response from
   http://nextcloud.oxigen.sg/.well-known/acme-challenge/Pm1_52BfpjoXrdKUEe4STSd4B8FpB9_bACu969qyBQ8
   [116.202.30.75]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.
10 

The nextcloud directory is owned by apache.

root@nextcloud:~# ls -al /var/www/
total 16
drwxr-xr-x  4 root     root     4096 Jan  9 10:29 .
drwxr-xr-x 12 root     root     4096 Jan  9 10:01 ..
drwxr-xr-x  2 root     root     4096 Jan  9 10:02 html
drwxr-xr-x 15 www-data www-data 4096 Mar 29 09:56 nextcloud
11

Show me /var/www/nextcloud/.htaccess

12 

<IfModule mod_headers.c>
  <IfModule mod_setenvif.c>
    <IfModule mod_fcgid.c>
       SetEnvIfNoCase ^Authorization$ "(.+)" XAUTHORIZATION=$1
       RequestHeader set XAuthorization %{XAUTHORIZATION}e env=XAUTHORIZATION
    </IfModule>
    <IfModule mod_proxy_fcgi.c>
       SetEnvIfNoCase Authorization "(.+)" HTTP_AUTHORIZATION=$1
    </IfModule>
  </IfModule>

  <IfModule mod_env.c>
    # Add security and privacy related headers
    Header always set Referrer-Policy "no-referrer"
    Header always set X-Content-Type-Options "nosniff"
    Header always set X-Download-Options "noopen"
    Header always set X-Frame-Options "SAMEORIGIN"
    Header always set X-Permitted-Cross-Domain-Policies "none"
    Header always set X-Robots-Tag "none"
    Header always set X-XSS-Protection "1; mode=block"
    SetEnv modHeadersAvailable true
  </IfModule>

  # Add cache control for static resources
  <FilesMatch "\.(css|js|svg|gif)$">
    Header set Cache-Control "max-age=15778463"
  </FilesMatch>

  # Let browsers cache WOFF files for a week
  <FilesMatch "\.woff2?$">
    Header set Cache-Control "max-age=604800"
 </FilesMatch>
</IfModule>
<IfModule mod_php7.c>
  php_value mbstring.func_overload 0
  php_value default_charset 'UTF-8'
  php_value output_buffering 0
  <IfModule mod_env.c>
    SetEnv htaccessWorking true
  </IfModule>
</IfModule>
<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteCond %{HTTP_USER_AGENT} DavClnt
  RewriteRule ^$ /remote.php/webdav/ [L,R=302]
  RewriteRule .* - [env=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
  RewriteRule ^\.well-known/host-meta /public.php?service=host-meta [QSA,L]
  RewriteRule ^\.well-known/host-meta\.json /public.php?service=host-meta-json [QSA,L]
  RewriteRule ^\.well-known/webfinger /public.php?service=webfinger [QSA,L]
  RewriteRule ^\.well-known/nodeinfo /public.php?service=nodeinfo [QSA,L]
  RewriteRule ^\.well-known/carddav /remote.php/dav/ [R=301,L]
  RewriteRule ^\.well-known/caldav /remote.php/dav/ [R=301,L]
  RewriteRule ^remote/(.*) remote.php [QSA,L]
  RewriteRule ^(?:build|tests|config|lib|3rdparty|templates)/.* - [R=404,L]
  RewriteCond %{REQUEST_URI} !^/\.well-known/(acme-challenge|pki-validation)/.*
  RewriteRule ^(?:\.|autotest|occ|issue|indie|db_|console).* - [R=404,L]
</IfModule>
<IfModule mod_mime.c>
  AddType image/svg+xml svg svgz
  AddEncoding gzip svgz
</IfModule>
<IfModule mod_dir.c>
  DirectoryIndex index.php index.html
</IfModule>
AddDefaultCharset utf-8
Options -Indexes
<IfModule pagespeed_module>
  ModPagespeed Off
</IfModule>
#### DO NOT CHANGE ANYTHING ABOVE THIS LINE ####

ErrorDocument 403 //
ErrorDocument 404 //
13

you need to remove \.| from that line.

14

Removed as instructed. Restarted Apache2 then certbot --apache. Same error.

Failed authorization procedure. nextcloud.oxigen.sg (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://nextcloud.oxigen.sg/.well-known/acme-challenge/onkdDEyw5zlA-qIQeEdwQOiixSzw7XKwZW1Phs9id0Q [116.202.30.75]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: nextcloud.oxigen.sg
   Type:   unauthorized
   Detail: Invalid response from
   http://nextcloud.oxigen.sg/.well-known/acme-challenge/onkdDEyw5zlA-qIQeEdwQOiixSzw7XKwZW1Phs9id0Q
   [116.202.30.75]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.
15

Is nextcloud resetting it?

You might need to use webroot with another directory.
create a /var/www/certbot directory, then

Add this to your relevant virtualhosts:

<Location "/.well-known/acme-challenge">
  DocumentRoot /var/www/certbot
</Location>

and use this command:

certbot -a webroot -w /var/www/certbot -i apache
16

Sorry, did not get an email notification on your latest reply.

Error after the above.
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Error while running apache2ctl configtest.
Action 'configtest' failed.
The Apache error log may have more information.

AH00526: Syntax error on line 16 of /etc/apache2/sites-enabled/nextcloud-le-ssl.conf:
DocumentRoot not allowed in <Location> context

The apache plugin is not working; there may be problems with your existing configuration.
The error was: MisconfigurationError("Error while running apache2ctl configtest.\nAction 'configtest' failed.\nThe Apache error log may have more information.\n\nAH00526: Syntax error on line 16 of /etc/apache2/sites-enabled/nextcloud-le-ssl.conf:\nDocumentRoot not allowed in <Location> context\n")

Below shows where I inserted the location tags

<IfModule mod_ssl.c>
<VirtualHost *:443>
        # The ServerName directive sets the request scheme, hostname and port t$
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        ServerName nextcloud.oxigen.sg

        ServerAdmin chris.chan@runbox.com
        DocumentRoot /var/www/nextcloud

        <Location "/.well-known/acme-challenge">
          DocumentRoot /var/www/certbot
        </Location>

        <IfModule mod_headers.c>
                Header always set Strict-Transport-Security "max-age=15552000; $
        </IfModule>

        <Directory /var/www/nextcloud/>
                AllowOverride All
        </Directory>


        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf


SSLCertificateFile /etc/letsencrypt/live/nextcloud.oxigen.sg/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/nextcloud.oxigen.sg/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>
17

The location block I suggested you to add… You should have put it in the port 80 virtualhost.

I was wrong, you should have used the Alias directive instead of DocumentRoot: https://httpd.apache.org/docs/2.4/mod/mod_alias.html#alias

<Location "/.well-known/acme-challenge">
    Alias "/var/www/certbot"
</Location>
18

Changed to alias for virtualhost 80. still 404 error.

Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for nextcloud.oxigen.sg
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. nextcloud.oxigen.sg (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://nextcloud.oxigen.sg/.well-known/acme-challenge/vOlqA4qZqF4gT34liwk1HIsTHJNJqXGIRULqN2lXWMA [116.202.30.75]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: nextcloud.oxigen.sg
   Type:   unauthorized
   Detail: Invalid response from
   http://nextcloud.oxigen.sg/.well-known/acme-challenge/vOlqA4qZqF4gT34liwk1HIsTHJNJqXGIRULqN2lXWMA
   [116.202.30.75]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.
19

404 is kinda fine, it’s those goddamn assholes that decided it was a proper error code to send for unauthorized access that make me mad.

try putting a file in /var/www/certbot/ and check if you can see it on

http://nextcloud.oxigen.sg/.well-known/acme-challenge/
or
http://nextcloud.oxigen.sg/.well-known/acme-challenge/.well-known/acme-challenge/

20

Added test.html to /var/www/certbot
The 2 links goes to nextcloud.
Should the test file be in /var/www/certbot/.well-known/acme-challenge/?

Maybe this will help. I installed nextcloud using this. https://nerdonthestreet.com/wiki?find=Install+Nextcloud+17+on+Debian+10

Then I installed collabora using these.


collabora.oxigen.sg auto renews LE perfectly