Failed authorization procedure

JohnnyMustang · February 5, 2019, 5:56pm

Hi, I’m having trouble with Certbot. I had a bit of trouble before, and I managed to fix it by looking at the virtual host files and editing. I want to run the command again to be able to redirect all requests as HTTPS. I didn’t do it at first because I wasn’t sure if it’d work, but now it does. Any idea what I’m doing wrong here? Thanks!

My domain is: loadscreen.net

I ran this command: sudo certbot --apache -d loadscreen.net -d www.loadscreen.net

It produced this output: Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache

You have an existing certificate that contains a portion of the domains you
requested (ref: /etc/letsencrypt/renewal/loadscreen.net.conf)

It contains these names: loadscreen.net

You requested these names for the new certificate: loadscreen.net,
www.loadscreen.net.

Do you want to expand and replace this existing certificate with the new
certificate?

(E)xpand/©ancel: E
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for loadscreen.net
http-01 challenge for www.loadscreen.net
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. www.loadscreen.net (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.loadscreen.net/.well-known/acme-challenge/dYv-Bgj-y5B-mg8-m9rXicysLE0O8uxSE2GvMtpLUqI: "\n<html lang=“en”>\n \n <meta charset=“utf-8”>\n <meta name=“viewport” content=“width=device-width, init”

IMPORTANT NOTES:

The following errors were reported by the server:

Domain: www.loadscreen.net
Type: unauthorized
Detail: Invalid response from
http://www.loadscreen.net/.well-known/acme-challenge/dYv-Bgj-y5B-mg8-m9rXicysLE0O8uxSE2GvMtpLUqI:
"\n<html lang=“en”>\n \n <meta
charset=“utf-8”>\n <meta name=“viewport”
content=“width=device-width, init”

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

The operating system my web server runs on is (include version): Ubuntu 18.04

My hosting provider, if applicable, is: Digitalocean

I can login to a root shell on my machine (yes or no, or I don’t know): Yes:

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.28.0

JuergenAuer · February 5, 2019, 6:09pm

Hi @JohnnyMustang

it's curious. Checking your domain ( https://check-your-website.server-daten.de/?q=loadscreen.net )

Domainname	Http-Status	redirect	Sec.	G
• http://www.loadscreen.net/
104.248.115.65	301	http://loadscreen.net/	0.237	D

• http://loadscreen.net/
104.248.115.65	200		0.497	H

• https://www.loadscreen.net/
104.248.115.65	301	https://loadscreen.net/	6.080	N
Certificate error: RemoteCertificateNameMismatch

• https://loadscreen.net/
104.248.115.65	200		6.537	B

• http://www.loadscreen.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
104.248.115.65	301	http://loadscreen.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de	0.243	D

• http://loadscreen.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
104.248.115.65	404		0.473	A
Not Found

there is a redirect www -> non-www, so your www should work if your non-www is ok.

Try to find your DocumentRoot (this value in your VirtualHost). Then use this value:

certbot run -a webroot -i apache -w yourDocumentRoot -d loadscreen.net -d www.loadcreen.net

JohnnyMustang · February 5, 2019, 6:13pm

This is the output I get:

root@LoadScreen:/etc/apache2/sites-available# certbot run -a webroot -i apache -w /var/www/html -d loadscreen.net -d www.loadcreen.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer apache

You have an existing certificate that contains a portion of the domains you
requested (ref: /etc/letsencrypt/renewal/loadscreen.net.conf)

It contains these names: loadscreen.net

You requested these names for the new certificate: loadscreen.net,
www.loadcreen.net.

Do you want to expand and replace this existing certificate with the new
certificate?

(E)xpand/(C)ancel: E
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for loadscreen.net
http-01 challenge for www.loadcreen.net
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. www.loadcreen.net (http-01): urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up A for www.loadcreen.net

IMPORTANT NOTES:

The following errors were reported by the server:

Domain: www.loadcreen.net
Type: None
Detail: DNS problem: NXDOMAIN looking up A for www.loadcreen.net
root@LoadScreen:/etc/apache2/sites-available#

JuergenAuer · February 5, 2019, 6:14pm

I had a typo - www.loadcreen.net - your domain is loadscreen.net

JohnnyMustang · February 5, 2019, 6:15pm

Ah, damn. Didn't realise, probably should have since I'm getting a rate limit error now. Any idea how long should I wait before attempting again?

JuergenAuer · February 5, 2019, 6:16pm

You had tried it 5 times. This is the rate limit, wait one hour.

JohnnyMustang · February 5, 2019, 6:17pm

Alright, will do. Thanks for your help.

JohnnyMustang · February 5, 2019, 7:46pm

I tried it now and it worked. Thanks so much for your help!

This is the output:

root@LoadScreen:/etc/apache2/sites-available# certbot run -a webroot -i apache -w /var/www/html -d loadscreen.net -d www.loadscreen.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer apache

You have an existing certificate that contains a portion of the domains you
requested (ref: /etc/letsencrypt/renewal/loadscreen.net.conf)

It contains these names: loadscreen.net

You requested these names for the new certificate: loadscreen.net,
www.loadscreen.net.

Do you want to expand and replace this existing certificate with the new
certificate?

(E)xpand/(C)ancel: E
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for loadscreen.net
http-01 challenge for www.loadscreen.net
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/loadscreen.net-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/loadscreen.net-le-ssl.conf

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.

1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.

Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Redirecting vhost in /etc/apache2/sites-enabled/loadscreen.net.conf to ssl vhost in /etc/apache2/sites-enabled/loadscreen.net-le-ssl.conf

Your existing certificate has been successfully renewed, and the new certificate
has been installed.

The new certificate covers the following domains: https://loadscreen.net and
https://www.loadscreen.net

You should test your configuration at:
SSL Server Test: loadscreen.net (Powered by Qualys SSL Labs)
SSL Server Test (Powered by Qualys SSL Labs)

IMPORTANT NOTES:

Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/loadscreen.net/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/loadscreen.net/privkey.pem
Your cert will expire on 2019-05-06. To obtain a new or tweaked
version of this certificate in the future, simply run certbot again
with the "certonly" option. To non-interactively renew all of
your certificates, run "certbot renew"

If you like Certbot, please consider supporting our work by:

Donating to ISRG / Let's Encrypt: Donate - Let's Encrypt
Donating to EFF: Support EFF's Work on Let's Encrypt | Electronic Frontier Foundation

From what I've gathered, the path was somehow wrong. Is that right? I'm sure the document root was correct, though.

JohnnyMustang · February 5, 2019, 8:00pm

HTTPS links are failing now.

This is what I get on the browser:

Not Found

The requested URL /2018/09/05/nintendo-direct-announced-likely-to-include-pokemon-lets-go/ was not found on this server.

But it works with http. Any ideas?

rg305 · February 5, 2019, 8:05pm

hmm...

Why twice?

Please review the file for correctness.

rg305 · February 5, 2019, 8:06pm

The http and the https sites have different root folders.
There are conflicting/overlapping names in the configs.

JohnnyMustang · February 5, 2019, 8:07pm

How would I go about fixing it?

rg305 · February 5, 2019, 8:10pm

You need to find the problem first.

[should give a general idea of what is going on]

JohnnyMustang · February 5, 2019, 8:13pm

I see. This is what I get from that command:

/etc/apache2/sites-enabled/000-default.conf:<VirtualHost *:80>
/etc/apache2/sites-enabled/000-default.conf: ServerAdmin admin@loadscreen.net
/etc/apache2/sites-enabled/000-default.conf: ServerName loadscreen.net
/etc/apache2/sites-enabled/000-default.conf: ServerAlias www.loadscreen.net
/etc/apache2/sites-enabled/000-default.conf: DocumentRoot /var/www/html
/etc/apache2/sites-enabled/000-default.conf:
/etc/apache2/apache2.conf:# supposed to determine listening ports for incoming connections which can be
/etc/apache2/apache2.conf:# ServerRoot: The top of the directory tree under which the server's
/etc/apache2/apache2.conf:#ServerRoot "/etc/apache2"
/etc/apache2/apache2.conf:ServerName localhost
/etc/apache2/apache2.conf:# If you do not specify an ErrorLog directive within a
/etc/apache2/apache2.conf:# logged here. If you do define an error logfile for a
/etc/apache2/apache2.conf:# Include list of ports to listen on
/etc/apache2/apache2.conf:# not allow access to the root filesystem outside of /usr/share and /var/www.
/etc/apache2/conf-available/security.conf:# If you use version control systems in your document root, you should
/etc/apache2/conf-available/other-vhosts-access-log.conf:# Define an access log for VirtualHosts that don't define their own logfile
/etc/apache2/conf-available/localized-error-pages.conf:# even on a per-VirtualHost basis. If you include the Alias in the global server
/etc/apache2/ports.conf:# have to change the VirtualHost statement in
/etc/apache2/ports.conf:Listen 80
/etc/apache2/ports.conf: Listen 443
/etc/apache2/ports.conf: Listen 443
/etc/apache2/mods-available/userdir.conf: UserDir disabled root
/etc/apache2/mods-available/info.conf: # http://servername/server-info (requires that mod_info.c be loaded).
/etc/apache2/mods-available/cache_disk.conf: CacheRoot /var/cache/apache2/mod_cache_disk
/etc/apache2/mods-available/status.conf: # with the URL of http://servername/server-status
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf:<VirtualHost *:443>
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf: ServerAdmin admin@loadscreen.net
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf: ServerName loadscreen.net
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf: ServerAlias www.loadscreen.net
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf:SSLCertificateFile /etc/letsencrypt/live/loadscreen.net/fullchain.pem
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf:SSLCertificateKeyFile /etc/letsencrypt/live/loadscreen.net/privkey.pem
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf:
/etc/apache2/sites-available/loadscreen.net.conf:<VirtualHost *:80>
/etc/apache2/sites-available/loadscreen.net.conf: ServerAdmin admin@loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf: ServerName loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf: ServerAlias www.loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/loadscreen.net.conf:RewriteCond %{SERVER_NAME} =www.loadscreen.net [OR]
/etc/apache2/sites-available/loadscreen.net.conf:RewriteCond %{SERVER_NAME} =loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf:
/etc/apache2/sites-available/default-ssl.conf:
/etc/apache2/sites-available/default-ssl.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/default-ssl.conf:
/etc/apache2/sites-available/000-default.conf:<VirtualHost *:80>
/etc/apache2/sites-available/000-default.conf: ServerAdmin admin@loadscreen.net
/etc/apache2/sites-available/000-default.conf: ServerName $domain
/etc/apache2/sites-available/000-default.conf: ServerAlias www.$domain
/etc/apache2/sites-available/000-default.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/000-default.conf:
/etc/apache2/sites-available/000-default.conf.dpkg-dist:<VirtualHost *:80>
/etc/apache2/sites-available/000-default.conf.dpkg-dist: # The ServerName directive sets the request scheme, hostname and port that
/etc/apache2/sites-available/000-default.conf.dpkg-dist: # redirection URLs. In the context of virtual hosts, the ServerName
/etc/apache2/sites-available/000-default.conf.dpkg-dist: #ServerName www.example.com
/etc/apache2/sites-available/000-default.conf.dpkg-dist: DocumentRoot /var/www/html
/etc/apache2/sites-available/000-default.conf.dpkg-dist:

JuergenAuer · February 5, 2019, 8:14pm

Happy to read that.

Now you have one certificate with two domain names:

CN=loadscreen.net
	05.02.2019
	06.05.2019
	loadscreen.net, www.loadscreen.net - 2 entries

Both connections are secure.

Yep, then your http and your https VirtualHost have different DocumentRoots.

DocumentRoot /var/www/html

Check your SSL-config and use the same DocumentRoot.

rg305 · February 5, 2019, 8:20pm

Please show:
ls -l /etc/apache2/sites-enabled/
[to understand exactly which configs are in use]

In the meantime…

These seems redundant:
[or they may be the “same” file]

/etc/apache2/sites-enabled/000-default.conf:<VirtualHost *:80>
/etc/apache2/sites-enabled/000-default.conf: ServerName loadscreen.net
/etc/apache2/sites-enabled/000-default.conf: ServerAlias www.loadscreen.net
/etc/apache2/sites-enabled/000-default.conf: DocumentRoot /var/www/html
/etc/apache2/sites-enabled/000-default.conf:</VirtualHost>

/etc/apache2/sites-available/loadscreen.net.conf:<VirtualHost *:80>
/etc/apache2/sites-available/loadscreen.net.conf: ServerName loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf: ServerAlias www.loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/loadscreen.net.conf:</VirtualHost>

Please show this file:
/etc/apache2/sites-available/default-ssl.conf
[don’t know what that is doing]

This one seems… “unused”:

/etc/apache2/sites-available/000-default.conf:<VirtualHost *:80>
/etc/apache2/sites-available/000-default.conf: ServerName $domain
/etc/apache2/sites-available/000-default.conf: ServerAlias www.$domain
/etc/apache2/sites-available/000-default.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/000-default.conf:</VirtualHost>

JohnnyMustang · February 5, 2019, 8:21pm

This is my loadscreen.net.conf file

And this is loadscreen.net-le-ssl.conf file

ServerAdmin admin@loadscreen.net ServerName loadscreen.net ServerAlias www.loadscreen.net DocumentRoot /var/www/html ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/loadscreen.net/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/loadscreen.net/privkey.pem

JohnnyMustang · February 5, 2019, 8:24pm

ServerAdmin webmaster@localhost

            DocumentRoot /var/www/html

            ErrorLog ${APACHE_LOG_DIR}/error.log
            CustomLog ${APACHE_LOG_DIR}/access.log combined

           <FilesMatch "\.(cgi|shtml|phtml|php)$">
                            SSLOptions +StdEnvVars
            </FilesMatch>
            <Directory /usr/lib/cgi-bin>
                            SSLOptions +StdEnvVars
            </Directory>

    </VirtualHost>

rg305 · February 5, 2019, 8:28pm

Not related - Not in use.

/etc/apache2/sites-enabled/000-default.conf
conflicts with:
/etc/apache2/sites-enabled/loadscreen.net.conf > /etc/apache2/sites-available/loadscreen.net.conf

rg305 · February 5, 2019, 8:34pm

Please show:
grep -Ei 'include|optional' /etc/apache2