Unable to Renew Three Sites

Help
#1

My domain is: https://quantum-equities.com

I ran this command: # certbot certonly --csr /etc/letsencrypt/csr-quantum-equities.com.csr --fullchain-path /etc/pki/tls/chains/quantum-equities.com_fullchain-2018-02-19.pem --chain-path /etc/pki/tls/chains/quantum-equities.com_chain-2018-02-19.pem --cert-path /etc/pki/tls/certs/quantum-equities.com_cert-2018-02-19.pem

It produced this output: Failed authorization procedure. mail.quantum-equities.com (tls-sni-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Timeout

My web server is (include version): Apache 2.4.6 67.el7.centos.6

The operating system my web server runs on is (include version): CentOS 7.4

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site: No

I’ve made a script to do automatic renewals (below) which did not work on this first round. Manually entering the command (above) produced an error that certbot could not confirm any of my sites. It is a mystery why, as the old cert is still valid and my mail sites ping. (I have ping turned off for web)

Obviously mail.quantum-equities.com is alive and well.

As an aside, it seems pretty clear at this point that RHEL’s SELinux implementation is very broken with certbot, so I just turn it off in the script. I see plenty of SELinux errors when certbot attempts to run, to wit:
type=AVC msg=audit(1519075254.297:146): avc: denied { write } for pid=2185 comm=“httpd” path="/var/lib/letsencrypt/.certbot.lock" dev=“dm-0” ino=33818389 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=0
type=AVC msg=audit(1519075254.297:146): avc: denied { write } for pid=2185 comm=“httpd” path="/var/log/letsencrypt/.certbot.lock" dev=“dm-0” ino=407083 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:var_log_t:s0 tclass=file permissive=0
type=AVC msg=audit(1519075254.297:146): avc: denied { write } for pid=2185 comm=“httpd” path="/etc/letsencrypt/.certbot.lock" dev=“dm-0” ino=16839603 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:etc_t:s0 tclass=file permissive=0
type=AVC msg=audit(1519075254.354:147): avc: denied { write } for pid=2188 comm=“httpd” path="/var/lib/letsencrypt/.certbot.lock" dev=“dm-0” ino=33818389 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=0
type=AVC msg=audit(1519075254.354:147): avc: denied { write } for pid=2188 comm=“httpd” path="/var/log/letsencrypt/.certbot.lock" dev=“dm-0” ino=407083 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:var_log_t:s0 tclass=file permissive=0
type=AVC msg=audit(1519075254.354:147): avc: denied { write } for pid=2188 comm=“httpd” path="/etc/letsencrypt/.certbot.lock" dev=“dm-0” ino=16839603 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:etc_t:s0 tclass=file permissive=0

So with all the problems to sort through I just turn off SELinux during the cert renewal script. But it still can not complete. This will be an emergency soon.

/usr/local/bin/letsencrypt-renew

#!/bin/sh

DATE=$(date +%Y-%m-%d)
DIR=/etc/pki/tls

setenforce 0

certbot handling

first it cannot replace certs, so ensure new locations (date suffix)

each time mean the certificate is unique each time. Next, it’s

really chatty, so the only way to tell if there was a failure is to

check whether the certificates got updated and then get cron to

email the log

Quantum-Equities.com

DIR2=/etc/letsencrypt/live/quantum-equities.com
CERT="${DIR}/certs/quantum-equities.com_cert-${DATE}.pem"
FULLCHAIN="${DIR}/chains/quantum-equities.com_fullchain-${DATE}.pem"
CHAIN="${DIR}/chains/quantum-equities.com_chain-${DATE}.pem"
CSR=/etc/letsencrypt/csr-quantum-equities.com.csr
OUT=/tmp/certbot-QE.out

certbot certonly --csr ${CSR} --fullchain-path ${FULLCHAIN} --chain-path ${CHAIN} --cert-path ${CERT} --apache > ${OUT} 2>&1

if [ ! -f ${FULLCHAIN} -o ! -f ${CHAIN} -o ! -f ${CERT} ]; then
cat /tmp/certbot-QE.out | mail -s “TLS Cert Update Fail for quantum-equities.com” postmaster@quantum-equities.com
fi

link into place

cert only (apache needs)

ln -sf ${CERT} ${DIR2}/cert.pem

cert with chain (stunnel needs)

ln -sf ${FULL} ${DIR2}/fullchain.pem

chain only (apache needs)

ln -sf ${CHAIN} ${DIR}/chain.pem

Quantum-Sci.com

DIR2=/etc/letsencrypt/live/quantum-sci.com
CERT="${DIR}/certs/quantum-sci.com_cert-${DATE}.pem"
FULLCHAIN="${DIR}/chains/quantum-sci.com_fullchain-${DATE}.pem"
CHAIN="${DIR}/chains/quantum-sci.com_chain-${DATE}.pem"
CSR=/etc/letsencrypt/csr-quantum-sci.com.csr
OUT=/tmp/certbot-QS.out

certbot certonly --csr ${CSR} --fullchain-path ${FULLCHAIN} --chain-path ${CHAIN} --cert-path ${CERT} --apache > ${OUT} 2>&1

if [ ! -f ${FULLCHAIN} -o ! -f ${CHAIN} -o ! -f ${CERT} ]; then
cat ${OUT} | mail -s “TLS Cert Update Fail for quantum-sci.com” postmaster@quantum-sci.com
fi

link into place

cert only (apache needs)

ln -sf ${CERT} ${DIR2}/cert.pem

cert with chain (stunnel needs)

ln -sf ${FULL} ${DIR2}/fullchain.pem

chain only (apache needs)

Quantum-Sci.com

DIR2=/etc/letsencrypt/live/quantum-sci.com
CERT="${DIR}/certs/quantum-sci.com_cert-${DATE}.pem"
FULLCHAIN="${DIR}/chains/quantum-sci.com_fullchain-${DATE}.pem"
CHAIN="${DIR}/chains/quantum-sci.com_chain-${DATE}.pem"
CSR=/etc/letsencrypt/csr-quantum-sci.com.csr
OUT=/tmp/certbot-QS.out

certbot certonly --csr ${CSR} --fullchain-path ${FULLCHAIN} --chain-path ${CHAIN} --cert-path ${CERT} --apache > ${OUT} 2>&1

if [ ! -f ${FULLCHAIN} -o ! -f ${CHAIN} -o ! -f ${CERT} ]; then
cat ${OUT} | mail -s “TLS Cert Update Fail for quantum-sci.com” postmaster@quantum-sci.com
fi

link into place

cert only (apache needs)

ln -sf ${CERT} ${DIR2}/cert.pem

cert with chain (stunnel needs)

ln -sf ${FULL} ${DIR2}/fullchain.pem

chain only (apache needs)

ln -sf ${CHAIN} ${DIR}/chain.pem

Delphi-Real-Estate.com

DIR2=/etc/letsencrypt/live/delphi-real-estate.com
CERT="${DIR}/certs/delphi-real-estate.com_cert-${DATE}.pem"
FULLCHAIN="${DIR}/chains/delphi-real-estate.com_fullchain-${DATE}.pem"
CHAIN="${DIR}/chains/delphi-real-estate.com_chain-${DATE}.pem"
CSR=/etc/letsencrypt/csr-delphi-real-estate.com.csr
OUT=/tmp/certbot-DRE.out

certbot certonly --csr ${CSR} --fullchain-path ${FULLCHAIN} --chain-path ${CHAIN} --cert-path ${CERT} --apache > ${OUT} 2>&1

if [ ! -f ${FULLCHAIN} -o ! -f ${CHAIN} -o ! -f ${CERT} ]; then
cat ${OUT} | mail -s “TLS Cert Update Fail for delphi-real-estate.com” postmaster@delphi-real-estate.com
fi

link into place

cert only (apache needs)

ln -sf ${CERT} ${DIR2}/cert.pem

cert with chain (stunnel needs)

ln -sf ${FULL} ${DIR2}/fullchain.pem

chain only (apache needs)

ln -sf ${CHAIN} ${DIR}/chain.pem

setenforce 1

reload the services

systemctl reload httpd
#systemctl restart stunnel4

#2

Hi,

It seems the mail server and the other two domains are not in the same server.

For certbot being able to verify, all three domains need to be in same server.

Currently,
The two of your website are @ 50.47.99.2

Your mail server is @ 72.251.232.108

Thank you.

#3

But, you are correct about the IPs. This is where they are. I can’t change that.

#4

Hi,

Can you clarify what u are trying to do?

I’m confused.

Thank you

#5

I have a web server (quantum-equities.com) at 50.47.99.2 and a mailserver (mail.quantum-equities.com) at 72.251.232.108. I use the same certs for both, but for now I just want to fix web, although the SAN must contain all domains.

This is my first cert renewal and all my certs have now expired so I have no mail or web and this is an emergency. Cert renewal is not supposed to depend on IP, but on the sites I put in the SAN.

Because of security measures put in the DNS record, I don’t want to renew the cert every two months. I want to use a CSR to renew the same cert for about a year or so.

My renewal script is on a systemd timer which triggers a service that calls this script:

#!/bin/bash -x
# If this fails in January, check to see if symlinks have changed in /etc/letsencrypt/live/{domain}/
# Culprit is likely selinux.

DATE=$(date +%Y-%m-%d)
DIR=/etc/pki/tls

setenforce 0

##
# certbot handling
#
# first it cannot replace certs, so ensure new locations (date suffix)
# each time mean the certificate is unique each time.  Next, it's
# really chatty, so the only way to tell if there was a failure is to
# check whether the certificates got updated and then get cron to
# email the log
##

# Quantum-Equities.com
DIR2=/etc/letsencrypt/live/quantum-equities.com
CERT="${DIR}/certs/quantum-equities.com_cert-${DATE}.pem"
FULLCHAIN="${DIR}/chains/quantum-equities.com_fullchain-${DATE}.pem"
CHAIN="${DIR}/chains/quantum-equities.com_chain-${DATE}.pem"
CSR=/etc/letsencrypt/csr-quantum-equities.com.csr
OUT=/tmp/certbot-QE.out

certbot certonly --csr ${CSR} --fullchain-path ${FULLCHAIN} --chain-path ${CHAIN} --cert-path ${CERT} --apache > ${OUT} 2>&1

if [ ! -f ${FULLCHAIN} -o ! -f ${CHAIN} -o ! -f ${CERT} ]; then
    cat /tmp/certbot-QE.out | mail -s "TLS Cert Update Fail for quantum-equities.com" postmaster@quantum-equities.com
fi

# link into place

# cert only (apache needs)
ln -sf ${CERT} ${DIR2}/cert.pem
# cert with chain (stunnel needs)
ln -sf ${FULL} ${DIR2}/fullchain.pem
# chain only (apache needs)
ln -sf ${CHAIN} ${DIR}/chain.pem
#6

Hi,

Your mail server is having trouble renew because there’s no http or https port open.(port 80 and 443 are both closed)

I’m just trying to confirm:
Do you have all domains in one server? Or is in separate server?

Because the certbot command I saw above only works if you have all domains included in the cert in one server.

If the mail. Is in a separate server, please use a csr only contains the mail domian and try run the command.

Thank you.

#7

If your web and mail are on different server. I’m wondering how you issued it in command line first time…

#8

They were on the same domain before, but I discovered that incoming 25 is blocked. So I had to move mail to a different server. Both http ports are open incoming.

I can manually move the cert to the mail server after renewal. It’s a question of passing whatever this test is for domains so I can get renewed. I’d understood that no IP testing is involved.

Should I add a virtual domain in Apache for mail.quantum-equities.com? Oh no, because it won’t resolve there.

Ideally I’d like my cert acquirer on its own machine. Then it would gather the right certs and distribute them with scp or something. Is it necessary that certbot get the web certs on the actual machine that serves web. And separately on the actual machine that serves mail?

#9

Hi,

(personally) i think it's ideal to seperate your mail server from web. (aka use a dedicated cert on mail server and a cert on your web server)
Since in this case you can avoid transfer problems.

Thank you

1 Like
#10

But this means that there can be no dedicated cert-getting machine? (No local ‘cert pseudo-authority’ which only job is to get certs?)

That each server must fetch its own cert renewals?

Is this solely because of tests that certbot runs?

#11

Yes.
Because if the two domains are on different serers, it's hard to get a new certificate using any validation methods (except DNS-01).

Thank you

#12

On the mail server:
# certbot certonly -d mail.quantum-equities.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log

How would you like to authenticate with the ACME CA?
-------------------------------------------------------------------------------
1: Apache Web Server plugin - Beta (apache)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)
-------------------------------------------------------------------------------
Select the appropriate number [1-3] then [enter] (press 'c' to cancel): 2
Plugins selected: Authenticator standalone, Installer None
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
tls-sni-01 challenge for mail.quantum-equities.com
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. mail.quantum-equities.com (tls-sni-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Timeout

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: mail.quantum-equities.com
   Type:   connection
   Detail: Timeout

Of course it’s not going to be able to confirm an Apache server on the mail server. What does it want now?

#13

You'll need to open port 443 too as it's using TLS-SNI

Thank you.

1 Like
#14

Both the HTTP-01 and DNS-01 validation methods have mechanisms available to delegate authority to obtain certificates to another machine, so you can have a dedicated cert-getting machine.

For HTTP-01 there does need to be a web server listening on the machine that the certificate is for, but it can return an HTTP 301 redirect for /.well-known/acme-challenge to a different web server, and the certificate authority will follow that redirect. So, the HTTP 301 redirect serves as a way to delegate authority to obtain certificates for that machine.

For DNS-01 you can have a cert-getting machine with API credentials to update DNS records, including DNS records related to other machines. If you don't want to or can't use an API this way for your main DNS zone(s), you can create DNS CNAME records (for the special subdomain _acme-challenge) that point to names in another DNS zone (which could be on a completely separate domain) and then give the cert-getting machine API credentials to update records in the target DNS zone. Here, the DNS CNAME record serves as a way to delegate authority to obtain certificates.

I think I should write a web page about these methods, with examples, because this is a very frequent question on this forum!

Edit: Because of security concerns, the TLS-SNI-01 method has been disabled for new issuance and is only available for renewals. Unlike the other methods, there is no way to delegate it to another machine.

3 Likes
#15

schoen, it is good to have this information, but I’m not familiar with a number of the terms you use. (HTTP-01, DNS-01, /.well-known/acme-challenge, subdomain _acme-challenge, etc) Probably I just don’t know the mechanisms of certbot.

I’ve saved this in my notes in case someday I have time to sort it out.

#16

https://certbot.eff.org/docs/challenges.html
https://certbot.eff.org/docs/using.html#getting-certificates-and-choosing-plugins

Some of this documentation needs to be updated in light of the deprecation of TLS-SNI-01 and the addition of better DNS-01 support in Certbot.

#17

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.