Why can't I certbot -d toastmastervoice.com and create a new VirtualHost and certificate?

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: toastmastervoice.com

I ran this command: certbot-auto -d toastmastervoice.com

It produced this output: At my second attempt, which reported the same “Could not reverse map the HTTPS VirtualHost to the original”:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Cert not yet due for renewal

You have an existing certificate that has exactly the same domains or certificate name you requested and isn’t close to expiry.
(ref: /etc/letsencrypt/renewal/toastmastervoice.com.conf)

What would you like to do?

1: Attempt to reinstall this existing certificate
2: Renew & replace the cert (may be subject to CA rate limits)

Select the appropriate number [1-2] then [enter] (press ‘c’ to cancel): 1
Keeping the existing certificate
Could not reverse map the HTTPS VirtualHost to the original

IMPORTANT NOTES:

  • Unable to install the certificate
  • Congratulations! Your certificate and chain have been saved at:
    /etc/letsencrypt/live/toastmastervoice.com/fullchain.pem
    Your key file has been saved at:
    /etc/letsencrypt/live/toastmastervoice.com/privkey.pem
    Your cert will expire on 2020-11-17. To obtain a new or tweaked
    version of this certificate in the future, simply run certbot-auto
    again with the “certonly” option. To non-interactively renew all
    of your certificates, run “certbot-auto renew”

My web server is (include version):

Server version: Apache/2.4.38 (Debian)
Server built: 2019-10-15T19:53:42
The operating system my web server runs on is (include version):

Debian Buster

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don’t know):

Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):

No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):

Certbot 1.7.0

2

What’s the output of apachectl -S?

3

I tried to post it directly and I received a censoring message saying that new users are not allowed to post more than 120 links in a post. I posted the output at https://cjshayward.com/wp-content/project/apachectl-S.txt

4

I’ve posted the information you requested, providing a link to another site rather than the direct output because it has more than 120 FQDN’s and people only at my level of seniority do not have the privilege of posting as many links as the output of an apachectl -S. Could you review the output you requested, now at https://cjshayward.com/wp-content/project/apachectl-S.txt?

5

Sorry! I skimmed the file on my mobile phone earlier, but forgot about it afterwards… My apologies.

Looks like certbot worked in the past, looking at the numerous HTTPS sites which look generated from the HTTP configuration file by certbot. Nothing really stands out which could make certbot fail.

Could you perhaps also upload the /var/log/letsencrypt/letsencrypt.log file after certbot failed? It might give more information (but I’m not sure how many things about the Apache installation part is logged…)

6

Thanks, and no worries.

The whole logfile is too large for me to post, but I’ve posted it to http://toastmastervoice.com/letsencryptlog.txt.

Thanks,

7

Although that log file is also interesting, because a few things are going very wrongly, this isn't – I think – the log file after you issued the command you posted in the first post of this thread. The reason why I think both is, among others, the last line of the log file:

2020-08-20 12:08:58,724:ERROR:certbot._internal.log:4 renew failure(s), 2 parse failure(s)

This makes me think this was one of the certbot renew commands perhaps ran by a systemd timer or cronjob. Also, it shows 4 renewal failures (due to different things) and 2 renewal configuration files certbot can't parse.

However, those problems listed above probably aren't related to your current issue. It does make me worry about the general health of your servers configuration.

9

There are existing entries for sites that are no longer live.

Would you recommend, for server health, going through and removing all entries that correspond to a no longer live site?

Thanks,

10

@freessltools.com That’s because the standalone plugin is used while a webserver is running. And not related to the current issue.

@CJSHayward In my opinion, keeping a clean working environment without any clutter is always a good thing. But like I said, it probably isn’t the cause of your current problem. For your current problem, we really need the log from when certbot tried to get a certificate for the problematic domain.

11

Absolutely. Repeated failures are a huge waste of everyone's resources.

12

@Osiris

Ah. Good eye. I just followed back from the cascade of errors and came to the binding issue.

13

Thank you. I made another trial run of certbot-auto -d toastmastervoice.org, and got the same output. I then copied the log to http://toastmastervoice.com/letsencryptlog2.txt - does that have better information?

14

I'm not sure what @Osiris will see, but the following stands out to me:

2020-08-20 16:17:45,604:
DEBUG:certbot._internal.cert_manager:
Renewal conf file /etc/letsencrypt/renewal/cjsh.name-0001.conf is broken. Skipping.
...
CertStorageError: renewal config file {} is missing a required file reference

2020-08-20 16:17:45,606:
DEBUG:certbot._internal.cert_manager:
Renewal conf file /etc/letsencrypt/renewal/cjsh.name.conf is broken. Skipping.
...
CertStorageError: renewal config file {} is missing a required file reference

16

It does look like a problem with the Certbot Apache plugin. During certificate installation, it seems unable to generate an HTTPS version of your virtualhost.

I don’t think the other errors are related.

Could you paste the contents of:

/etc/apache2/sites-enabled/089-toastmasters.conf

Hopefully that will be enough to diagnose it. If not, we might need your entire Apache configuration…

18

They are for unrelated certificates. I don’t see how they could have any impact on the domain in question.

20

Certbot goes through all of the existing lineages to determine whether there is one that already covers toastmastervoice.com, or whether it needs to create a new lineage.

While it is going through the lineages, it finds that some of them are broken (config file is bad, or symlinks are messed up, or whatever). When Certbot encounters a broken lineage, it logs the error, skips it and moves onto the next one.

Finally, it reaches the end, and it hasn’t found a matching lineage, so it decides to create a new one for toastmastervoice.com. In this log, it does find an existing one, and asks the user whether they want to re-issue the existing certificate or re-install it (the latter choice is made).

At this point, the problems with the other lineages are probably quite irrelevant. We’re not using them, we don’t care about them. One broken lineage isn’t going to (at least, doesn’t IME) make the rest of Certbot blow up.

21

@_az

Ah. Thanks for the great explanation. Makes sense. This is why I’m a big fan of divorcing the various aspects of the process from one another. I guess in my eyes I’d rather deal with a different machine (with a known-good configuration) handling the transaction process and clerical work. I’ve seen too many cases lately of configuration changes made by certbot conflicting with particular setups. Not that that’s necessarily what has happened in this topic. Guess I feel like there’s a difference between getting certificates properly issued and configuring a system to properly use https.

22

/etc/apache2/sites-enabled/089-toastmasters.conf is:

<VirtualHost *:80>
	ServerAdmin christos.hayward@gmail.com
    ServerName toastmastervoice.com
	ServerAlias www.toastmastervoice.com toastmastersvoice.com www.toastmastersvoice.com
    #ServerAlias www.orthodoxchurchfathers.com fathers.jonathanscorner.com

	DocumentRoot /home/christos/toastmasters/
	<Directory />
		Options FollowSymLinks
		AllowOverride None
	</Directory>
	<Directory /home/christos/toastmasters>
		Options ExecCGI FollowSymLinks Indexes MultiViews
		AllowOverride All
		Order allow,deny
		allow from all
	</Directory>

	DirectoryIndex index.php index.cgi index.html

	ErrorLog ${APACHE_LOG_DIR}/toastmasters-error.log

	# Possible values include: debug, info, notice, warn, error, crit,
	# alert, emerg.
	LogLevel warn

	CustomLog ${APACHE_LOG_DIR}/toastmasters-access.log combined

    Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
        Allow from 127.0.0.0/255.0.0.0 ::1/128
    </Directory>

#RewriteEngine on
#RewriteCond %{SERVER_NAME} =orthodoxchurchfathers.cjshayward.com [OR]
#RewriteCond %{SERVER_NAME} =orthodoxchurchfathers.com
#RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

    #<VirtualHost *:80>
	#ServerAdmin CJSHayward@POBox.com
	#ServerName www.orthodoxchurchfathers.com
	#ServerAlias fathers.jonathanscorner.com
	#DocumentRoot /home/christos/oldmirror
	#RewriteEngine On
	#RewriteRule ^(.*)$ http://orthodoxchurchfathers.com$1 [R=301,L]
#RewriteCond %{SERVER_NAME} =fathers.jonathanscorner.com [OR]
#RewriteCond %{SERVER_NAME} =www.orthodoxchurchfathers.com
#RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
    #</VirtualHost>
23

Things don’t work out in the following certbot function:

I don’t actually fully understand the code exactly, but perhaps (and that’s a big “perhaps”) it’s because the VirtualHost includes three ServerAlias options which are not included in the command you’ve used? You only specified toastmastervoice.com while the VirtualHost also specifies www.toastmastervoice.com, toastmastersvoice.com and www.toastmastersvoice.com.

You could try including all four hostnames (with four times the -d option).

24

I haven’t had ServerAlias cause difficulties before, and I don’t really suspect it now.

I tried, for a sanity check, to comment out all the ServerAliases leaving only the ServerName entry. The behavior when I tried to reinstall the existing certificate (instead of the #2 option of Request and Renew) seemed to be identical.

I might be able to get the desired behavior by adapting and installing the certificate according to the pattern of the other sites-enabled entries, but that’s a losing solution.