Migrating to new server

mirko77 · June 13, 2017, 4:24pm

HI there,

We have a domain using a certificate installed with LetsEncrypt, at five.epicollect.net

Now we are migrating to a new server, which is live but with a temporary domain name. (http://fi–ec5dev3.dide.ic.ac.uk/)

I am trying to generate a new certificate on the new server covering both domains, so when we point five.epicollect.net to the new server https will still work.

I tried, from the new server fi–ec5dev3.dide.ic.ac.uk:

sudo certbot --apache -d five.epicollect.net -d fi--ec5dev3.dide.ic.ac.uk

but it fails with:

Failed authorization procedure. five.epicollect.net (tls-sni-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Incorrect validation certificate for tls-sni-01 challenge. Requested 695b7c76c82623839355b212f327af59.f5eb69fb808f144b95c3674b88d87f59.acme.invalid from 129.31.26.230:443. Received 2 certificate(s), first certificate had names “five.epicollect.net”

IMPORTANT NOTES:

The following errors were reported by the server:

Domain: five.epicollect.net
Type: unauthorized
Detail: Incorrect validation certificate for tls-sni-01 challenge.
Requested
695b7c76c82623839355b212f327af59.f5eb69fb808f144b95c3674b88d87f59.acme.invalid
from 129.31.26.230:443. Received 2 certificate(s), first
certificate had names “five.epicollect.net”

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A record(s) for that domain
contain(s) the right IP address.

Any suggestions?

rg305 · June 13, 2017, 4:28pm

Name: five.epicollect.net
Address: 129.31.26.230

Name: fi–ec5dev3.dide.ic.ac.uk
Address: 129.31.26.11

Without some special consideration, you won’t be able to auth both names from one IP.

mirko77 · June 13, 2017, 4:56pm

Yes, they are on two different IPs address.

Would I have any option? What do you mean by special consideration?

I have ownership of both servers, can I perform the authorization manually?

rg305 · June 13, 2017, 4:59pm

DNS auth would work from any IP (multiple IPs).
Auth redirection [301] would also work - forward auth requests from domain1 to domain2.

mirko77 · June 13, 2017, 5:21pm

Would the manual plugin work? https://certbot.eff.org/docs/using.html#manual

schoen · June 13, 2017, 5:25pm

Yes, but I think the auth redirection method is better!

If you set an HTTP 301 redirect from /.well-known/acme-challenge on the old machine to the same location on the new machine, then the new machine can pass HTTP-01 validation for both names, e.g. using the Certbot --webroot method.

mirko77 · June 13, 2017, 5:31pm

Thanks,

I was looking at the webroot as well here https://certbot.eff.org/docs/using.html#webroot

But it does not mention any 301 redirect, though. It asks me to specify the web root of each server where the temporary verification file will be placed.

I suppose both methods will work, right?

schoen · June 13, 2017, 5:34pm

Hi @mirko77,

The documentation assumes that Certbot will be run on the web server where the DNS record is already pointing. The situation where DNS records for two or more subject names point to different servers, and/or you want to obtain a certificate for a name which points to a server other than the one where you’re running Certbot, is considered a much more advanced case, which I guess currently isn’t mentioned in the documentation at all.

The thing that makes me not recommend --manual is that certbot renew cannot renew certificates obtained with it (at least if you literally use it in the manual mode).

mirko77 · June 13, 2017, 6:44pm

Thanks for the explanation @schoen I will give the 301 redirect a try and report back. Thanks @rg305 as well

mirko77 · June 14, 2017, 11:17am

Hi @schoen
Here is an update.

I set the redirection, you can test it going to https://five.epicollect.net/.well-known/acme-challenge/ will redirect you to http://fi–ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/

I tried a few things, the file test.txt is accessible, folders have 755 permission.

When I run
sudo certbot certonly --webroot -w /var/www/html_prod/shared/public/ -d five.epicollect.net -d fi--ec5dev3.dide.ic.ac.uk

I get the following error

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for five.epicollect.net
http-01 challenge for fi--ec5dev3.dide.ic.ac.uk
Using the webroot path /var/www/html_prod/shared/public for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. fi--ec5dev3.dide.ic.ac.uk (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://fi--ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/BOHzIjplae56vjC7B_vopSTV5-0wOaO_7P8CTDmeICE: "<!DOCTYPE html>
<html>
<head>
        <meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="v", five.epicollect.net (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://five.epicollect.net/.well-known/acme-challenge/rEJVv8HOl48SSAQfjKm2GPCnHHFA8HKjIqEuVclMs-4: "<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
 <head>
  <title>Index of /.well-known/acme-challenge</title>
 </"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: fi--ec5dev3.dide.ic.ac.uk
   Type:   unauthorized
   Detail: Invalid response from
   http://fi--ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/BOHzIjplae56vjC7B_vopSTV5-0wOaO_7P8CTDmeICE:
   "<!DOCTYPE html>
   <html>
   <head>
           <meta charset="utf-8">
   <meta http-equiv="X-UA-Compatible" content="IE=edge">
   <meta name="v"

   Domain: five.epicollect.net
   Type:   unauthorized
   Detail: Invalid response from
   http://five.epicollect.net/.well-known/acme-challenge/rEJVv8HOl48SSAQfjKm2GPCnHHFA8HKjIqEuVclMs-4:
   "<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
   <html>
    <head>
     <title>Index of /.well-known/acme-challenge</title>
    </"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address.

I tried removing the directory listing, I placed an index.html instead, still no luck.

Now I am locked out after a few attempts
There were too many requests of a given type :: Error creating new authz :: Too many invalid authorizations recently.
so I need to wait 1 hour at least, right?

edit: If I remove .well-know/acme-challenge folders from fi--ec5dev3.dide.ic.ac.uk certbot does not create them and I get a 404

edit 2:
using curl
curl -i http://fi--ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/test | cat -A

response is

HTTP/1.1 200 OK^M$
Date: Wed, 14 Jun 2017 14:17:11 GMT^M$
Server: Apache/2.4.18 (Ubuntu)^M$
Last-Modified: Wed, 14 Jun 2017 14:04:17 GMT^M$
ETag: "8-551ec072ba3a8"^M$
Accept-Ranges: bytes^M$
Content-Length: 8^M$
^M$
success$

rg305 · June 14, 2017, 3:16pm

This seem to work.

But also everything else in that folder redirects to the other FOLDER.
So, http://five.epicollect.net/.well-known/acme-challenge/test.txt
returns
http://fi–ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/
NOT
http://fi–ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/test.txt
as expected.

Please show the redirect statement.

mirko77 · June 14, 2017, 8:15pm

Hi @rg305 thanks for spotting that, I fixed the redirect.

I am using an .htaccess file with:

RewriteEngine on
RewriteRule ^(.*)$ http://fi--ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/$1 [R=301]

Not the redirection appears to work, but I still get a 404:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for fi--ec5dev3.dide.ic.ac.uk
http-01 challenge for five.epicollect.net
Using the webroot path /var/www/html_prod/shared/public for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. five.epicollect.net (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://five.epicollect.net/.well-known/acme-challenge/EB0tCZQhn9kQlsoOuojuiTXqoiz-NrqPQOmoxNkk5zE: "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p", fi--ec5dev3.dide.ic.ac.uk (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://fi--ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/8SMtvsSKu2PG1xlDhzjg6-_1ZX9fG5t-51hKRaU8lic: "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: five.epicollect.net
   Type:   unauthorized
   Detail: Invalid response from
   http://five.epicollect.net/.well-known/acme-challenge/EB0tCZQhn9kQlsoOuojuiTXqoiz-NrqPQOmoxNkk5zE:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   Domain: fi--ec5dev3.dide.ic.ac.uk
   Type:   unauthorized
   Detail: Invalid response from
   http://fi--ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/8SMtvsSKu2PG1xlDhzjg6-_1ZX9fG5t-51hKRaU8lic:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address.

rg305 · June 14, 2017, 8:43pm

you don’t really show enough of your coding to be sure…
But it may be that:
RewriteEngine on
RewriteRule ^(.*)$ http://fi–ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/$1 [R=301]

turns this:
https://five.epicollect.net/.well-known/acme-challenge/xxx

into this:
http://fi–ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/.well-known/acme-challenge/xxx

schoen · June 14, 2017, 8:44pm

To avoid this problem, you could also test with --staging.

mirko77 · June 14, 2017, 9:41pm

Hi @rg305,

I do not have any other code to show you, I only have a .htaccess with

RewriteEngine on
RewriteRule ^(.*)$ http://fi--ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/$1 [R=301]

on five.epicollect.net, my main domain, which has got already a LetsEncrypt certificate.

The redirection works fine, have you tried?

if you open the http://five.epicollect.net/.well-known/acme-challenge/xxx in your browser, you are redirected to http://fi–ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/xxx, and a 404 error because obviously xxx does not exist. All as expected.

If you do the same with http://five.epicollect.net/.well-known/acme-challenge/test, it will redirect correctly to http://fi–ec5dev3.dide.ic.ac.uk/.well-known/acme-challenge/test showing “success” on the page.

On my main domain five.epicollect.net, there is a redirection from http to https since there is already a certificate installed, could that be the problem?

It looks like Cerbot cannot create the temporary files for the verification.

@schoen, I suppose the manual mode would be my only option here?

schoen · June 14, 2017, 9:54pm

Are you doing the redirection in the right direction? That is, are you still running Certbot on fi–ec5dev3?

mirko77 · June 14, 2017, 9:56pm

Yes, I am on fi–ec5dev3, with sudo rights.

The redirection is set on five.epicollect.net, the old domain

schoen · June 14, 2017, 10:05pm

Right now, your test.txt file is only in /var/www/html_prod/shared/public/.well-known/acme-challenge?

mirko77 · June 14, 2017, 10:22pm

on fi–ec5dev3, in /var/www/html_prod/current/public/.well-known/acme-challenge I have test and test.txt

on five.epicollect.net, in /var/www/html_prod/current/public/.well-known/acme-challenge I have the .htaccess with the 301 redirect

I am serving from /current/ as I have a simlink in place. I have exactly the same setup on both servers.

schoen · June 14, 2017, 10:28pm

Do you mean that, because of the symlink, /var/www/html_prod/shared/public and /var/www/html_prod/current/public are the same directory? Can you double-check that with

cat /var/www/html_prod/{current,shared}/public/.well-known/acme-challenge

?