Failed authorization procedure

Alex0h · November 20, 2019, 12:05pm

Hello!

I ran this command:
certbot --apache -d sovet-mgd.ru -d www.sovet-mgd.ru

It produced this output:

 - The following errors were reported by the server:

   Domain: www.sovet-mgd.ru
   Type:   unauthorized
   Detail: Invalid response from
   http://www.sovet-mgd.ru/.well-known/acme-challenge/-LF6HPMIIb5LUcTCHGO-KZDcsNz6CDaX-IP8mykBk6I
   [194.48.97.191]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   Domain: sovet-mgd.ru
   Type:   unauthorized
   Detail: Invalid response from
   http://sovet-mgd.ru/.well-known/acme-challenge/j9waVqynBZzQS9JuoI3pQ-YuSXsXkthEUqzP1fDvpwA
   [194.48.97.191]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

My web server is (include version): Apache/2.4.25 (Debian)

The operating system my web server runs on is (include version): Debian 9.5

My hosting provider, if applicable, is: 1cloud

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.28.0

rg305 · November 20, 2019, 12:25pm

Your Apache config may be confused or confusing certbot to the point it can’t tell exactly where to place the challenge files.

Please show the output of:
apachectl -S

[never fear, there is a fix to it - to all of it]

Alex0h · November 20, 2019, 12:32pm

VirtualHost configuration:
194.48.97.191:443      is a NameVirtualHost
         default server a-tsm.ru (/etc/apache2/vhosts/a-tsm/a-tsm.ru:19)
         port 443 namevhost a-tsm.ru (/etc/apache2/vhosts/a-tsm/a-tsm.ru:19)
                 alias b.a-tsm.ru
                 alias t.a-tsm.ru
                 alias www.a-tsm.ru
         port 443 namevhost legalclp.com (/etc/apache2/vhosts/a-tsm/legalclp.com:22)
                 alias www.legalclp.com
         port 443 namevhost niistandart.ru (/etc/apache2/vhosts/a-tsm/niistandart.ru:19)
                 alias nii.a-tsm.ru
                 alias www.niistandart.ru
         port 443 namevhost sovet-mgd.ru (/etc/apache2/vhosts/a-tsm/sovet-mgd.ru:19)
                 alias www.sovet-mgd.ru
194.48.97.191:80       is a NameVirtualHost
         default server a-tsm.ru (/etc/apache2/vhosts/a-tsm/a-tsm.ru:1)
         port 80 namevhost a-tsm.ru (/etc/apache2/vhosts/a-tsm/a-tsm.ru:1)
                 alias b.a-tsm.ru
                 alias t.a-tsm.ru
                 alias www.a-tsm.ru
         port 80 namevhost camarbitrale.it (/etc/apache2/vhosts/a-tsm/camarbitrale.it:1)
                 alias www.camarbitrale.it
         port 80 namevhost de-arbitrage.de (/etc/apache2/vhosts/a-tsm/de-arbitrage.de:1)
                 alias www.de-arbitrage.de
         port 80 namevhost interarbi.com (/etc/apache2/vhosts/a-tsm/interarbi.com:1)
                 alias www.interarbi.com
         port 80 namevhost legalclp.com (/etc/apache2/vhosts/a-tsm/legalclp.com:1)
                 alias www.legalclp.com
         port 80 namevhost niistandart.ru (/etc/apache2/vhosts/a-tsm/niistandart.ru:1)
                 alias nii.a-tsm.ru
                 alias www.niistandart.ru
         port 80 namevhost rsa.sg (/etc/apache2/vhosts/a-tsm/rsa.sg:1)
                 alias www.rsa.sg
         port 80 namevhost souz-u-t-s.ru (/etc/apache2/vhosts/a-tsm/souz-u-t-s.ru:1)
                 alias www.souz-u-t-s.ru
         port 80 namevhost sovet-mgd.ru (/etc/apache2/vhosts/a-tsm/sovet-mgd.ru:1)
                 alias www.sovet-mgd.ru
         port 80 namevhost ta-paris.fr (/etc/apache2/vhosts/a-tsm/ta-paris.fr:1)
                 alias www.ta-paris.fr
         port 80 namevhost vesarbitrazh.ru (/etc/apache2/vhosts/a-tsm/vesarbitrazh.ru:1)
                 alias www.vesarbitrazh.ru
         port 80 namevhost yantgorod-tsz.ru (/etc/apache2/vhosts/a-tsm/yantgorod-tsz.ru:1)
                 alias www.yantgorod-tsz.ru
*:80                   a-tsm.ru (/etc/apache2/sites-enabled/000-default.conf:1)
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex authdigest-opaque: using_defaults
Mutex watchdog-callback: using_defaults
Mutex rewrite-map: using_defaults
Mutex ssl-stapling-refresh: using_defaults
Mutex authdigest-client: using_defaults
Mutex ssl-stapling: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/run/apache2/" mechanism=default 
Mutex mpm-accept: using_defaults
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
Define: ENABLE_USR_LIB_CGI_BIN
User: name="www-data" id=33
Group: name="www-data" id=33

rg305 · November 20, 2019, 12:38pm

The problem should be found in here:

port 443 namevhost sovet-mgd.ru (/etc/apache2/vhosts/a-tsm/sovet-mgd.ru:19)
         alias www.sovet-mgd.ru
port 80 namevhost sovet-mgd.ru (/etc/apache2/vhosts/a-tsm/sovet-mgd.ru:1)
        alias www.sovet-mgd.ru
*:80    a-tsm.ru (/etc/apache2/sites-enabled/000-default.conf:1)

Please files:
/etc/apache2/sites-enabled/000-default.conf
/etc/apache2/vhosts/a-tsm/sovet-mgd.ru

Alex0h · November 20, 2019, 12:48pm

/etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:80>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        #ServerName www.example.com

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

/etc/apache2/vhosts/a-tsm/sovet-mgd.ru

<VirtualHost 194.48.97.191:80>
        ServerName sovet-mgd.ru
        ServerAlias www.sovet-mgd.ru
        DocumentRoot /var/www/a-tsm/data/www/sovet-mgd.ru
        ServerAdmin webmaster@sovet-mgd.ru
        DirectoryIndex index.php index.html
        AddDefaultCharset UTF-8
        AssignUserID a-tsm tsm
        CustomLog /var/www/httpd-logs/sovet-mgd.ru.access.log combined
        ErrorLog /var/www/httpd-logs/sovet-mgd.ru.error.log
        ScriptAlias /cgi-bin/ /var/www/a-tsm/data/www/sovet-mgd.ru/cgi-bin/
        <FilesMatch "\.ph(p[3-5]?|tml)$">
                SetHandler application/x-httpd-php5
        </FilesMatch>
        ScriptAlias /php-bin/ /var/www/php-bin-isp-php54/a-tsm/
        AddHandler application/x-httpd-php5 .php .php3 .php4 .php5 .phtml
        Action application/x-httpd-php5 /php-bin/php
</VirtualHost>
<VirtualHost 194.48.97.191:443>
        ServerName sovet-mgd.ru
        ServerAlias www.sovet-mgd.ru
        DocumentRoot /var/www/a-tsm/data/www/sovet-mgd.ru
        ServerAdmin webmaster@sovet-mgd.ru
        DirectoryIndex index.php index.html
        AddDefaultCharset UTF-8
        SSLEngine on
        SSLCertificateFile "/var/www/httpd-cert/a-tsm/sovet-mgd.ru_le1.crt"
        SSLCertificateKeyFile "/var/www/httpd-cert/a-tsm/sovet-mgd.ru_le1.key"
        SSLCertificateChainFile "/var/www/httpd-cert/a-tsm/sovet-mgd.ru_le1.ca"
        SSLHonorCipherOrder on
        SSLProtocol +TLSv1 +TLSv1.1 +TLSv1.2
        SSLCipherSuite EECDH:+AES256:-3DES:RSA+AES:!NULL:!RC4
        <IfModule headers_module>
                Header always set Strict-Transport-Security "max-age=31536000; preload"
        </IfModule>
        AssignUserID a-tsm tsm
        CustomLog /var/www/httpd-logs/sovet-mgd.ru.access.log combined
        ErrorLog /var/www/httpd-logs/sovet-mgd.ru.error.log
       ScriptAlias /cgi-bin/ /var/www/a-tsm/data/www/sovet-mgd.ru/cgi-bin/
        <FilesMatch "\.ph(p[3-5]?|tml)$">
                SetHandler application/x-httpd-php5
        </FilesMatch>
        ScriptAlias /php-bin/ /var/www/php-bin-isp-php54/a-tsm/
        AddHandler application/x-httpd-php5 .php .php3 .php4 .php5 .phtml
        Action application/x-httpd-php5 /php-bin/php
</VirtualHost>
<Directory /var/www/a-tsm/data/www/sovet-mgd.ru>
        Options +Includes +ExecCGI
        RewriteEngine on
        RewriteCond %{HTTPS} off
        RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R,L]
</Directory>

rg305 · November 20, 2019, 1:08pm

[the formatting was off on that last post & I ran out of edits - I’ll go back and delete it ASAP]

I don’t see the confusion.
It is not clear to me why certbot fails to use:
DocumentRoot /var/www/a-tsm/data/www/sovet-mgd.ru

But this is not a problem.
You have choices:

You can use --webroot to specify the location to be used:
certbot --webroot -w /var/www/a-tsm/data/www/sovet-mgd.ru -d sovet-mgd.ru -d www.sovet-mgd.ru
[this solution will probably be required for your other sites too]
You can define a global alias that will affect all sites (present and future) to use a specific challenge location:
Alias /.well-known/acme-challenge/ /some/path/you/like/
[I use: Alias /.well-known/acme-challenge/ /ACME-challenges/`]
[this will only require the one entry (in the main apache2.conf file) and the creation of that path]
[mkdir /some/path/you/like/]
You can use --standalone to have certbot use its’ own web server.
[this requires stopping your web service until validated - that affects all sites]

I recommend #2.
Any questions?

Alex0h · November 20, 2019, 1:15pm

Thank you for your answers.
I must create new path, for example /root/path/ ?
And add this string
Alias /.well-known/acme-challenge/ /root/path/
in end of file apache2.conf?
Am I right?
Thank you!

rg305 · November 20, 2019, 1:18pm

Yes.
Here is the tail of my Apache2.conf file:

IncludeOptional sites-enabled/*.conf
#send all ACME challenges to this dedicated location
Alias /.well-known/acme-challenge/ /ACME-challenges/

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

rg305 · November 20, 2019, 1:22pm

And the contents of that folder:

ls -la /ACME-challenges/
total 12
drwxr-xr-x  2 root root 4096 Nov  6 09:36 .
drwxr-xr-x 24 root root 4096 Nov 14 06:50 ..
-rw-r--r--  1 root root   41 Nov  6 07:30 test-file

The test-file is there to confirm access from the Internet:
http://<any-of-my-sites>/.well-known/ACME-challenges/test-file

JuergenAuer · November 20, 2019, 2:39pm

Hi @Alex0h

don't use ip addresses.

If there is another router or something else in front of that server, the wrong ip address is used.

Switch to

<VirtualHost *:80>

restart your server, then again apachectl -S.

Alex0h · November 21, 2019, 10:57am

Hello, i did as you suggested, but now I have Forbidden error.

   Domain: www.sovet-mgd.ru
   Type:   unauthorized
   Detail: Invalid response from
   http://www.sovet-mgd.ru/.well-known/acme-challenge/iJvif52pqXn8IoN4wejXdzgcIl7ZrR2JYEJSWN4B03g
   [194.48.97.191]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>403
   Forbidden</title>\n</head><body>\n<h1>Forbidden</h1>\n<p"

   Domain: sovet-mgd.ru
   Type:   unauthorized
   Detail: Invalid response from
   http://sovet-mgd.ru/.well-known/acme-challenge/bSjESbJc__0duO3rC9kjTlXz6nWru3FK3n-5KQpBPn0
   [194.48.97.191]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>403
   Forbidden</title>\n</head><body>\n<h1>Forbidden</h1>\n<p"

ls -la /f2c/ACME-challenges/ -my path
total 8
drwxr-xr-x 2 root root 4096 Nov 21 13:43 .
drwxr-xr-x 3 root root 4096 Nov 21 13:43 .

rg305 · November 21, 2019, 7:22pm

Place a test file in the new challenge folder:
echo 'test' > /f2c/ACME-challenges/testfile

Then see if it can be accessed from the Internet:
http://sovet-mgd.ru/.well-known/acme-challenge/testfile

Alex0h · November 25, 2019, 11:15am

Nope, forbidden

rg305 · November 25, 2019, 1:23pm

Then you need to allow access to that directory.

Try adding something like this to the apache2.conf file:

<Directory /f2c/ACME-challenges/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>

Alex0h · November 26, 2019, 7:37am

Still Forbidden

Alex0h · November 27, 2019, 9:15am

So I Have progress:

But error Not Found is back:

   Domain: www.sovet-mgd.ru
   Type:   unauthorized
   Detail: Invalid response from
   http://www.sovet-mgd.ru/.well-known/acme-challenge/NfD2cdP8V8kZzd03RzOh2bwZKjouIuTg4SJWnYzNZ1A
   [194.48.97.191]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   Domain: sovet-mgd.ru
   Type:   unauthorized
   Detail: Invalid response from
   http://sovet-mgd.ru/.well-known/acme-challenge/xhi6R_4hzAacF4GMTWqReVpKmeHSs9xg9nNShMT7QhE
   [194.48.97.191]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

JuergenAuer · November 27, 2019, 9:44am

Do you use --apache? Use --webroot with the directory you have created /.well-known/acme-challenge/testfile.

certbot -a webroot -w YourWebroot -i apache ...

https://certbot.eff.org/docs/using.html

Alex0h · November 27, 2019, 10:36am

certbot certonly --webroot --dry-run -w /var/www/a-tsm/data/www/sovet-mgd.ru -d sovet-mgd.ru -d www.sovet-mgd.ru

still “No found”

JuergenAuer · November 27, 2019, 10:38am

That's

your wrong webroot.

There

is the folder you have to use, the basic folder from /.well-known/acme-challenge

You are mixing different versions, that can't work. Choice - not both.

Alex0h · November 27, 2019, 10:45am

Is this right?
certbot certonly --webroot --dry-run -w /f2c/acme-challenges/ -d sovet-mgd.ru -d www.sovet-mgd.ru

   Domain: www.sovet-mgd.ru
   Type:   unauthorized
   Detail: Invalid response from
   http://www.sovet-mgd.ru/.well-known/acme-challenge/Rh-55ZmIKddceQrIW91InV89HgBJu2SFvayRJuIieBY
   [194.48.97.191]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"