Failed authorization procedure

Hi, I’m having trouble with Certbot. I had a bit of trouble before, and I managed to fix it by looking at the virtual host files and editing. I want to run the command again to be able to redirect all requests as HTTPS. I didn’t do it at first because I wasn’t sure if it’d work, but now it does. Any idea what I’m doing wrong here? Thanks!

My domain is: loadscreen.net

I ran this command: sudo certbot --apache -d loadscreen.net -d www.loadscreen.net

It produced this output: Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache


You have an existing certificate that contains a portion of the domains you
requested (ref: /etc/letsencrypt/renewal/loadscreen.net.conf)

It contains these names: loadscreen.net

You requested these names for the new certificate: loadscreen.net,
www.loadscreen.net.

Do you want to expand and replace this existing certificate with the new
certificate?


(E)xpand/©ancel: E
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for loadscreen.net
http-01 challenge for www.loadscreen.net
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. www.loadscreen.net (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.loadscreen.net/.well-known/acme-challenge/dYv-Bgj-y5B-mg8-m9rXicysLE0O8uxSE2GvMtpLUqI: "\n<html lang=“en”>\n \n <meta charset=“utf-8”>\n <meta name=“viewport” content=“width=device-width, init”

IMPORTANT NOTES:

The operating system my web server runs on is (include version): Ubuntu 18.04

My hosting provider, if applicable, is: Digitalocean

I can login to a root shell on my machine (yes or no, or I don’t know): Yes:

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.28.0

Hi @JohnnyMustang

it's curious. Checking your domain ( https://check-your-website.server-daten.de/?q=loadscreen.net )

Domainname Http-Status redirect Sec. G
http://www.loadscreen.net/
104.248.115.65 301 http://loadscreen.net/ 0.237 D
http://loadscreen.net/
104.248.115.65 200 0.497 H
https://www.loadscreen.net/
104.248.115.65 301 https://loadscreen.net/ 6.080 N
Certificate error: RemoteCertificateNameMismatch
https://loadscreen.net/
104.248.115.65 200 6.537 B
http://www.loadscreen.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
104.248.115.65 301 http://loadscreen.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.243 D
http://loadscreen.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
104.248.115.65 404 0.473 A
Not Found

there is a redirect www -> non-www, so your www should work if your non-www is ok.

Try to find your DocumentRoot (this value in your VirtualHost). Then use this value:

certbot run -a webroot -i apache -w yourDocumentRoot -d loadscreen.net -d www.loadcreen.net

This is the output I get:

root@LoadScreen:/etc/apache2/sites-available# certbot run -a webroot -i apache -w /var/www/html -d loadscreen.net -d www.loadcreen.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer apache


You have an existing certificate that contains a portion of the domains you
requested (ref: /etc/letsencrypt/renewal/loadscreen.net.conf)

It contains these names: loadscreen.net

You requested these names for the new certificate: loadscreen.net,
www.loadcreen.net.

Do you want to expand and replace this existing certificate with the new
certificate?


(E)xpand/(C)ancel: E
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for loadscreen.net
http-01 challenge for www.loadcreen.net
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. www.loadcreen.net (http-01): urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up A for www.loadcreen.net

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: www.loadcreen.net
    Type: None
    Detail: DNS problem: NXDOMAIN looking up A for www.loadcreen.net
    root@LoadScreen:/etc/apache2/sites-available#

I had a typo - www.loadcreen.net - your domain is loadscreen.net

Ah, damn. Didn't realise, probably should have since I'm getting a rate limit error now. Any idea how long should I wait before attempting again?

You had tried it 5 times. This is the rate limit, wait one hour.

Alright, will do. Thanks for your help.

I tried it now and it worked. Thanks so much for your help!

This is the output:

root@LoadScreen:/etc/apache2/sites-available# certbot run -a webroot -i apache -w /var/www/html -d loadscreen.net -d www.loadscreen.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer apache


You have an existing certificate that contains a portion of the domains you
requested (ref: /etc/letsencrypt/renewal/loadscreen.net.conf)

It contains these names: loadscreen.net

You requested these names for the new certificate: loadscreen.net,
www.loadscreen.net.

Do you want to expand and replace this existing certificate with the new
certificate?


(E)xpand/(C)ancel: E
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for loadscreen.net
http-01 challenge for www.loadscreen.net
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/loadscreen.net-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/loadscreen.net-le-ssl.conf

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.


1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.


Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Redirecting vhost in /etc/apache2/sites-enabled/loadscreen.net.conf to ssl vhost in /etc/apache2/sites-enabled/loadscreen.net-le-ssl.conf


Your existing certificate has been successfully renewed, and the new certificate
has been installed.

The new certificate covers the following domains: https://loadscreen.net and
https://www.loadscreen.net

You should test your configuration at:
SSL Server Test: loadscreen.net (Powered by Qualys SSL Labs)
SSL Server Test (Powered by Qualys SSL Labs)


IMPORTANT NOTES:

  • Congratulations! Your certificate and chain have been saved at:
    /etc/letsencrypt/live/loadscreen.net/fullchain.pem
    Your key file has been saved at:
    /etc/letsencrypt/live/loadscreen.net/privkey.pem
    Your cert will expire on 2019-05-06. To obtain a new or tweaked
    version of this certificate in the future, simply run certbot again
    with the "certonly" option. To non-interactively renew all of
    your certificates, run "certbot renew"

  • If you like Certbot, please consider supporting our work by:

    Donating to ISRG / Let's Encrypt: Donate - Let's Encrypt
    Donating to EFF: Support EFF's Work on Let's Encrypt | Electronic Frontier Foundation

From what I've gathered, the path was somehow wrong. Is that right? I'm sure the document root was correct, though.

HTTPS links are failing now.

This is what I get on the browser:

Not Found

The requested URL /2018/09/05/nintendo-direct-announced-likely-to-include-pokemon-lets-go/ was not found on this server.

But it works with http. Any ideas?

hmm...

Why twice?

Please review the file for correctness.

  • The http and the https sites have different root folders.
  • There are conflicting/overlapping names in the configs.

How would I go about fixing it?

You need to find the problem first.

Try:
grep -Eri 'loadscreen|root|listen|virtualhost|servername|serveralias' /etc/apache2

[should give a general idea of what is going on]

I see. This is what I get from that command:

/etc/apache2/sites-enabled/000-default.conf:<VirtualHost *:80>
/etc/apache2/sites-enabled/000-default.conf: ServerAdmin admin@loadscreen.net
/etc/apache2/sites-enabled/000-default.conf: ServerName loadscreen.net
/etc/apache2/sites-enabled/000-default.conf: ServerAlias www.loadscreen.net
/etc/apache2/sites-enabled/000-default.conf: DocumentRoot /var/www/html
/etc/apache2/sites-enabled/000-default.conf:
/etc/apache2/apache2.conf:# supposed to determine listening ports for incoming connections which can be
/etc/apache2/apache2.conf:# ServerRoot: The top of the directory tree under which the server's
/etc/apache2/apache2.conf:#ServerRoot "/etc/apache2"
/etc/apache2/apache2.conf:ServerName localhost
/etc/apache2/apache2.conf:# If you do not specify an ErrorLog directive within a
/etc/apache2/apache2.conf:# logged here. If you do define an error logfile for a
/etc/apache2/apache2.conf:# Include list of ports to listen on
/etc/apache2/apache2.conf:# not allow access to the root filesystem outside of /usr/share and /var/www.
/etc/apache2/conf-available/security.conf:# If you use version control systems in your document root, you should
/etc/apache2/conf-available/other-vhosts-access-log.conf:# Define an access log for VirtualHosts that don't define their own logfile
/etc/apache2/conf-available/localized-error-pages.conf:# even on a per-VirtualHost basis. If you include the Alias in the global server
/etc/apache2/ports.conf:# have to change the VirtualHost statement in
/etc/apache2/ports.conf:Listen 80
/etc/apache2/ports.conf: Listen 443
/etc/apache2/ports.conf: Listen 443
/etc/apache2/mods-available/userdir.conf: UserDir disabled root
/etc/apache2/mods-available/info.conf: # http://servername/server-info (requires that mod_info.c be loaded).
/etc/apache2/mods-available/cache_disk.conf: CacheRoot /var/cache/apache2/mod_cache_disk
/etc/apache2/mods-available/status.conf: # with the URL of http://servername/server-status
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf:<VirtualHost *:443>
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf: ServerAdmin admin@loadscreen.net
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf: ServerName loadscreen.net
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf: ServerAlias www.loadscreen.net
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf:SSLCertificateFile /etc/letsencrypt/live/loadscreen.net/fullchain.pem
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf:SSLCertificateKeyFile /etc/letsencrypt/live/loadscreen.net/privkey.pem
/etc/apache2/sites-available/loadscreen.net-le-ssl.conf:
/etc/apache2/sites-available/loadscreen.net.conf:<VirtualHost *:80>
/etc/apache2/sites-available/loadscreen.net.conf: ServerAdmin admin@loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf: ServerName loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf: ServerAlias www.loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/loadscreen.net.conf:RewriteCond %{SERVER_NAME} =www.loadscreen.net [OR]
/etc/apache2/sites-available/loadscreen.net.conf:RewriteCond %{SERVER_NAME} =loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf:
/etc/apache2/sites-available/default-ssl.conf:
/etc/apache2/sites-available/default-ssl.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/default-ssl.conf:
/etc/apache2/sites-available/000-default.conf:<VirtualHost *:80>
/etc/apache2/sites-available/000-default.conf: ServerAdmin admin@loadscreen.net
/etc/apache2/sites-available/000-default.conf: ServerName $domain
/etc/apache2/sites-available/000-default.conf: ServerAlias www.$domain
/etc/apache2/sites-available/000-default.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/000-default.conf:
/etc/apache2/sites-available/000-default.conf.dpkg-dist:<VirtualHost *:80>
/etc/apache2/sites-available/000-default.conf.dpkg-dist: # The ServerName directive sets the request scheme, hostname and port that
/etc/apache2/sites-available/000-default.conf.dpkg-dist: # redirection URLs. In the context of virtual hosts, the ServerName
/etc/apache2/sites-available/000-default.conf.dpkg-dist: #ServerName www.example.com
/etc/apache2/sites-available/000-default.conf.dpkg-dist: DocumentRoot /var/www/html
/etc/apache2/sites-available/000-default.conf.dpkg-dist:

Happy to read that.

Now you have one certificate with two domain names:

CN=loadscreen.net
	05.02.2019
	06.05.2019
	loadscreen.net, www.loadscreen.net - 2 entries

Both connections are secure.

Yep, then your http and your https VirtualHost have different DocumentRoots.

DocumentRoot /var/www/html

Check your SSL-config and use the same DocumentRoot.

Please show:
ls -l /etc/apache2/sites-enabled/
[to understand exactly which configs are in use]

In the meantime…

These seems redundant:
[or they may be the “same” file]

/etc/apache2/sites-enabled/000-default.conf:<VirtualHost *:80>
/etc/apache2/sites-enabled/000-default.conf: ServerName loadscreen.net
/etc/apache2/sites-enabled/000-default.conf: ServerAlias www.loadscreen.net
/etc/apache2/sites-enabled/000-default.conf: DocumentRoot /var/www/html
/etc/apache2/sites-enabled/000-default.conf:</VirtualHost>

/etc/apache2/sites-available/loadscreen.net.conf:<VirtualHost *:80>
/etc/apache2/sites-available/loadscreen.net.conf: ServerName loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf: ServerAlias www.loadscreen.net
/etc/apache2/sites-available/loadscreen.net.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/loadscreen.net.conf:</VirtualHost>

Please show this file:
/etc/apache2/sites-available/default-ssl.conf
[don’t know what that is doing]

This one seems… “unused”:

/etc/apache2/sites-available/000-default.conf:<VirtualHost *:80>
/etc/apache2/sites-available/000-default.conf: ServerName $domain
/etc/apache2/sites-available/000-default.conf: ServerAlias www.$domain
/etc/apache2/sites-available/000-default.conf: DocumentRoot /var/www/html
/etc/apache2/sites-available/000-default.conf:</VirtualHost>

This is my loadscreen.net.conf file

image

And this is loadscreen.net-le-ssl.conf file

ServerAdmin admin@loadscreen.net ServerName loadscreen.net ServerAlias www.loadscreen.net DocumentRoot /var/www/html ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined

Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/loadscreen.net/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/loadscreen.net/privkey.pem

ServerAdmin webmaster@localhost
            DocumentRoot /var/www/html

            ErrorLog ${APACHE_LOG_DIR}/error.log
            CustomLog ${APACHE_LOG_DIR}/access.log combined

           <FilesMatch "\.(cgi|shtml|phtml|php)$">
                            SSLOptions +StdEnvVars
            </FilesMatch>
            <Directory /usr/lib/cgi-bin>
                            SSLOptions +StdEnvVars
            </Directory>

    </VirtualHost>

Not related - Not in use.

/etc/apache2/sites-enabled/000-default.conf
conflicts with:
/etc/apache2/sites-enabled/loadscreen.net.conf > /etc/apache2/sites-available/loadscreen.net.conf

Please show:
grep -Ei 'include|optional' /etc/apache2