Random error while renewing or expanding: DNS problem?

danjde · February 16, 2017, 5:17pm

Hi friends,
On my VPS I’ve set up some Apache2 websites,
each of them use only the Record A (Adrress) for the DNS and not Record CNAME (Canonical NAME);

now, (not always!) when I try to renew or expand the single (overall) certificate obtain some errors as here:

Processing /etc/letsencrypt/renewal/server.sio4.org.conf
-------------------------------------------------------------------------------

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/server.sio4.org/fullchain.pem (failure)
IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: cosmogonia.org
   Type:   unauthorized
   Detail: Invalid response from
   http://cosmogonia.org/.well-known/acme-challenge/bDO4xUFgdIjt_xfioVhb_Yn--bgBclCQGYr6NSCVIFc:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   Domain: smtp.pergraziaricevuta.it
   Type:   unauthorized
   Detail: Invalid response from
   http://smtp.pergraziaricevuta.it/.well-known/acme-challenge/c14_PvffXM4b-xRJtWUK19H_act9DuBTm-h-w66CbKw:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   Domain: pop.pergraziaricevuta.it
   Type:   unauthorized
   Detail: Invalid response from
   http://pop.pergraziaricevuta.it/.well-known/acme-challenge/i5-4skb6nvR7ye2WBdyOV6cI39VssPyxNkUbr235iYA:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   Domain: www.vini-bulgarini.com
   Type:   unauthorized
   Detail: Invalid response from
   http://www.vini-bulgarini.com/.well-known/acme-challenge/ljv52nOIKFt1x4wyNgTyX0PQ-IzF7XiqCQxPR7-vB4Y:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   Domain: www.cosmogonia.org
   Type:   unauthorized
   Detail: Invalid response from
   http://www.cosmogonia.org/.well-known/acme-challenge/qFK6ZkvDnxyo9emWr0KD9FetMpaJ1SF9_o-49fTANDY:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   Domain: pop.cosmogonia.org
   Type:   unauthorized
   Detail: Invalid response from
   http://pop.cosmogonia.org/.well-known/acme-challenge/D-i8UGmMzYNIf6sFoweTsyBRVwazqnM7ewYr0hSZV00:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   Domain: pergraziaricevuta.it
   Type:   unauthorized
   Detail: Invalid response from
   http://pergraziaricevuta.it/.well-known/acme-challenge/WtiwyH2b47Wd4-_nccmoL8CaU3DiSP5l5UITCzWHRNs:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   Domain: server.sio4.org
   Type:   unauthorized
   Detail: Invalid response from
   http://server.sio4.org/.well-known/acme-challenge/eLIM7NnJfAP-veE6G_e0dyBMj_MO0IJJZrfLxIvslsM:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address.

Could it be caused by the reason that there is no CNAME as DNS record?

Many thanks!

schoen · February 16, 2017, 7:28pm

Hi @danjde, this error does not appear to be a DNS problem at all, but rather a problem with your web server configuration or else with Certbot’s ability to write the challenge files in the right place.

One typical question in this case is: if you put a text file in .well-known/acme-challenge inside your web root directory, are you able to view that text file over the web with a browser or with curl?

danjde · February 16, 2017, 8:17pm

Hi schoen and thanks for your reply!

I’ve created “.well-known/acme-challenge” directories and put into “prova.txt” with textual content “ciao!”

The answer to your question is: NO!
I’m unable to view the textual content (GASP!)

Shoud I remove the “.” from the “IndexIgnore” directive?

Osiris · February 16, 2017, 8:47pm

IndexIgnore is only used for automatically generated indexes for directories in Apache and can’t cause this.

Does the DocumentRoot directive in your Apache configuration files for the above domain(s) correspond with the directory in which you put the .well-known/acme-challenge/prova.txt file?

Also, I see that all requests with HTTP are all redirected to a host with HTTPS, including the pop.~ and smtp.~ hosts. It’s important to get the position of the test file (or challenge, in that case the webroot you entered when you got the certificate issued in the first place) right, because Let’s Encrypt will actually follow those redirects and expects to find the challenge in the new location.

danjde · February 17, 2017, 9:33am

If I create the same directory structure but without "initial dot" I can see the content of prova.txt: https://www.cosmogonia.org/well-known/acme-challenge/prova.txt

Then, I've seen that all my "http" Virtualhost had set the DocumentRoot, and now I've delete it as suggested here: RedirectSSL - HTTPD - Apache Software Foundation

At last, I do not know if this can answer your question, but looking into my DocumentRoot directives I've seen that it is not always the same, but I don't know if this could be a problem for the obtaining of certificates:

Here my DocumentRoot https for the FQDN:

/var/www/html

Here my DocumentRoot https for a standard website (`/var/www/website-domain/public_html/):

/var/www/cosmogonia.org/public_html/

Here my Letsencrypt Virtualhost:

Alias /.well-known/acme-challenge/ /var/www/letsencrypt/.well-known/acme-challenge/
<Directory "/var/www/letsencrypt/.well-known/acme-challenge/">
Options None
AllowOverride None
ForceType text/plain
# avoid access to anything not resembling a challenge
RedirectMatch 404 "^(?!/\.well-known/acme-challenge/[\w]{43}$)"
</Directory>

.. I hoping to understand your observations

Thanks!

Osiris · February 17, 2017, 1:36pm

You do realise the certificate you're requesting a certificate for has 22 FQDN's in it? At least that's what the certificate from server.sio4.org is giving me:

DNS:cosmogonia.org,
DNS:hotelsangiorgioriccione.com,
DNS:imap.cosmogonia.org,
DNS:imap.pergraziaricevuta.it,
DNS:mail.hotelsangiorgioriccione.com,
DNS:mail.sio4.org,
DNS:pergraziaricevuta.it,
DNS:pop.cosmogonia.org,
DNS:pop.pergraziaricevuta.it,
DNS:pop.sio4.org,
DNS:server.sio4.org,
DNS:sio4.org,
DNS:smtp.cosmogonia.org,
DNS:smtp.hotelsangiorgioriccione.com,
DNS:smtp.pergraziaricevuta.it,
DNS:smtp.sio4.org,
DNS:vini-bulgarini.com,
DNS:www.cosmogonia.org,
DNS:www.hotelsangiorgioriccione.com,
DNS:www.pergraziaricevuta.it,
DNS:www.sio4.org,
DNS:www.vini-bulgarini.com

And certbot tries to renew all those FQDN's.. So there's no such thing as a single "my Letsencrypt Virtualhost", unless you've got all those FQDN's as a ServerAlias in that single <VirtualHost>.

Anyway, back to your "VirtualHost".. I don't see a <VirtualHost> section.. Could it be that's because the forum stripped those things, thinking it's HTML? Or did you just not include it yourself? Because now I don't know if I'm looking at the HTTP VirtualHost, the HTTPS VirtualHost with a lot of things missing.. No clue.. Could be anything.

Also, you said the following: "Here my DocumentRoot https for the FQDN". Which FQDN? Like I said, there are 22 in your certificate! There's no such thing as the FQDN, unless you specify which FQDN you meant. For more information about the definition of a FQDN, see Fully qualified domain name - Wikipedia

What I'm seeing mostly here is, no offence, serious lack of understanding of how a webserver (in this case Apache) actually works, what the different directives actually do. You're pasting a piece of configuration file with an Alias directive in it. Have you read the Apache documentation about the Alias directive? It also has a RedirectMatch directive. You should read what that does too.

Also, you say you removed all the DocumentRoot directives. I hope you commented them, not delete them. Making permanent changes you don't actually know the ramifications of is unwise. A comment you can remove. Remembering what you deleted is a lot harder.
The page you're refering to called "RedirectSSL" has a Redirect directive in place. I'm not seeing such a directive in any of your configuration pieces you posted in this thread. (Spoiler, the RedirectMatch isn't actually redirecting anything.) So I have no idea if that was wise, possible or very unwise. I'm lacking that information.

Best thing for you to do is:

a) Read. Read alot. Read alot about managing a webserver in Apache;
b) Place all of your Apache configuration files on something like http://pastebin.com/ and post the links here.
c) Post the contents of /etc/letsencrypt/renewal/server.sio4.org.conf in this thread.

Perhaps then it's possible to sort things out.

danjde · February 20, 2017, 2:49pm

Yes, I know I do not know, quietly it does everything

Here two virtualhost domains http and https, the rest on the server are identical (if you want I can past if you want I can also show them):

HTTP sio4.org:
http://pastebin.com/uF3DDk3m
HTTPS sio4.org:
http://pastebin.com/ULiwC2RE
HTTP cosmogonia.org:
http://pastebin.com/hPxus01n
HTTPS cosmogonia.org:
http://pastebin.com/c2XkramL

Here the server.sio4.org.conf content:

http://pastebin.com/ULfvbcis

Many thanks for your help (if you want), useful in addition to indispensable reading to increase my knowledge and that of the reader

Osiris · February 20, 2017, 3:41pm

In post #5 you also posted some configuration pieces with Alias and <Directory> et cetera. Where did they go? I'm missing the configuration file where those things are mentioned.

danjde · February 20, 2017, 3:59pm

What you refer was /etc/apache2/sites-enabled/letsencrypt.conf:
http://pastebin.com/G18PiBys

I've not created that, I think it was created from an automatic (old) certbot debian installation. And I don't know if this is right to have it on sites-enabled..

Osiris · February 20, 2017, 4:11pm

You could try commenting the RedirectMatch in letsencrypt.conf.

danjde · February 20, 2017, 6:12pm

Hmmm! I’ve just commented out “RedirectMatch” and updated two DNS entries and added these to the certificate, using the “expand” option, and everything went smoothly without any error!

In the next days I will add other DNS entries and then will report!

For the moment, many thanks Osiris (especially for your patience!)

system · March 22, 2017, 6:12pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.