Renewed certificates, but apache and the live link cert is not the current one

cursulak · January 11, 2025, 7:11pm

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: Many (mfno.org, mcklpoa.ca, lawsonpark.ca, ursulak.ca, fundraiseukraine.ca)

I ran this command: certbot certonly --manual --preferred-challenges dns
All the domains are wildcards *.

It produced this output:

I get the acme code to update DNS
when verifying I sometimes get message cert2.pem already exists (not always)
see folder structure like this:

My web server is (include version): Apache 2.4.62

The operating system my web server runs on is (include version): MacOS Monterey

My hosting provider, if applicable, is: n/a, I run the server complete control.

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 3.0.1

There are very strange folder behaviours occurring and it's resulting in the incorrect symlink being formed for the new certificate. At one point I could recreate the links to the correct files, but this is not a sustainable solution and it has messed with certbot’s expected folder behaviour which I have not been able to get back on track.

When I renewed the current live symlink remains to the old (but still current files which expire in 10 days), the new ones are not linked. The new certs are however created for each domain when I run the command, this time as cert2.pem, chain2.pem... etc... in a whole mess of files ranging from cert1.pem up to cert6.pem.
How and were it keeps the all version numbers straight is being messed up.
At one point I blew the whole thing away and started over. Then in a matter of two updates, it was all messed up again.

Could someone help me understand

cursulak · January 11, 2025, 7:23pm

Also forgot to mention in the apache virtual hosts settings, a non version numbered path is set:

SSLEngine on
SSLCertificateFile /usr/local/etc/certbot/certs/live//fullchain.pem
SSLCertificateKeyFile /usr/local/etc/certbot/certs/live//privkey.pem

so I am presuming apaching is looking for the non-numbered version at that path to pull the correct one. If I rename files and move them around to accomplish this, it appears to prove true. However I don't feel that is supposed to be the correct behaviour I have to do each renewal.

cursulak · January 11, 2025, 7:26pm

it removed my coding in that post, between the double // is the domain name in the path.

MikeMcQ · January 11, 2025, 7:41pm

The way this is supposed to work is the in ../live/(cert-name) there are symlinks to the most recent set of cert files in ../archive/(cert-name)

When a new cert is issued its files go into /archive and the symlink in live is updated. You should always refer to the /live/ folder in Apache (or other references). Over the years Certbot has changed how much history it retains in /archive/. I believe it is now 6 (current and previous 5).

You should never manually copy/move files around in these folders. Certbot will manage them by itself.

The large variety of names in your /archive/ (-0002, -0011, ...) indicates lots of problems in the past. Certbot appends a number when something material has changed in the original cert profile but yet has some commonality to the original. Perhaps this was just you copying things around (incorrectly) but it usually points to a more fundamental problem.

The best place to sort this out is to show us output of this

sudo certbot certificates

Given you have many certs it that output is too long show these instead

sudo certbot certificates --cert-name mfno.org
sudo certbot certificates --cert-name mfno.org-0001
sudo certbot certificates --cert-name mfno.org-0002
sudo certbot certificates --cert-name mfno.org-0011

That is probably causing a lot of this grief. By copying files around you are probably damaging the symlinks.

cursulak · January 12, 2025, 12:25am

Thank you for the response. Here is the output to the general certbot certificates:

> 
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/barantas.ca-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/barantas.ca-0001/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/barantas.ca-0002.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/barantas.ca-0002/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/fundraiseukraine.ca-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/fundraiseukraine.ca-0001/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/fundraiseukraine.ca-0002.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/fundraiseukraine.ca-0002/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/lawsonpark.ca-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/lawsonpark.ca-0001/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/lawsonpark.ca-0002.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/lawsonpark.ca-0002/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/mcklpoa.ca-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/mcklpoa.ca-0001/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/mcklpoa.ca-0002.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/mcklpoa.ca-0002/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/mfno.org-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/mfno.org-0001/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/mfno.org-0002.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/mfno.org-0002/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/ursulak.ca-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/ursulak.ca-0001/cert.pem to be a symlink. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/ursulak.ca-0002.conf produced an unexpected error: renewal config file {} is missing a required file reference. Skipping.
> Renewal configuration file /usr/local/etc/certbot/certs/renewal/ursulak.ca-0003.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/ursulak.ca-0003/cert.pem to be a symlink. Skipping.
>

Found the following certs:

>   Certificate Name: barantas.ca
>     Serial Number: 3c45cc81e3cb3409ea60ddf5d557b57d57e
>     Key Type: ECDSA
>     Domains: *.barantas.ca
>     Expiry Date: 2025-01-27 23:20:12+00:00 (VALID: 15 days)
>     Certificate Path: /usr/local/etc/certbot/certs/live/barantas.ca/fullchain.pem
>     Private Key Path: /usr/local/etc/certbot/certs/live/barantas.ca/privkey.pem
>   Certificate Name: fundraiseukraine.ca
>     Serial Number: 454ece5defa32ae11df1d3e71bd0fd4797f
>     Key Type: ECDSA
>     Domains: *.fundraiseukraine.ca
>     Expiry Date: 2025-01-27 23:19:49+00:00 (VALID: 15 days)
>     Certificate Path: /usr/local/etc/certbot/certs/live/fundraiseukraine.ca/fullchain.pem
>     Private Key Path: /usr/local/etc/certbot/certs/live/fundraiseukraine.ca/privkey.pem
>   Certificate Name: lawsonpark.ca
>     Serial Number: 393a557adb83d2262a2f1bcdb57f1fefb27
>     Key Type: ECDSA
>     Domains: *.lawsonpark.ca
>     Expiry Date: 2025-01-27 23:17:10+00:00 (VALID: 15 days)
>     Certificate Path: /usr/local/etc/certbot/certs/live/lawsonpark.ca/fullchain.pem
>     Private Key Path: /usr/local/etc/certbot/certs/live/lawsonpark.ca/privkey.pem
>   Certificate Name: mcklpoa.ca
>     Serial Number: 4267b6c0977b7238f49c491c471b5fd7775
>     Key Type: ECDSA
>     Domains: *.mcklpoa.ca
>     Expiry Date: 2025-01-27 23:18:38+00:00 (VALID: 15 days)
>     Certificate Path: /usr/local/etc/certbot/certs/live/mcklpoa.ca/fullchain.pem
>     Private Key Path: /usr/local/etc/certbot/certs/live/mcklpoa.ca/privkey.pem
>   Certificate Name: mfno.org
>     Serial Number: 41a9bded0c1291bb23f46ffa70b9c141dca
>     Key Type: ECDSA
>     Domains: *.mfno.org
>     Expiry Date: 2025-01-27 23:19:17+00:00 (VALID: 15 days)
>     Certificate Path: /usr/local/etc/certbot/certs/live/mfno.org/fullchain.pem
>     Private Key Path: /usr/local/etc/certbot/certs/live/mfno.org/privkey.pem
>   Certificate Name: ursulak.ca
>     Serial Number: 460226a2242fb6feb83b0bfad8a6114d3ae
>     Key Type: ECDSA
>     Domains: *.ursulak.ca
>     Expiry Date: 2025-01-27 22:53:14+00:00 (VALID: 15 days)
>     Certificate Path: /usr/local/etc/certbot/certs/live/ursulak.ca/fullchain.pem
>     Private Key Path: /usr/local/etc/certbot/certs/live/ursulak.ca/privkey.pem

The following renewal configurations were invalid:

>   /usr/local/etc/certbot/certs/renewal/barantas.ca-0001.conf
>   /usr/local/etc/certbot/certs/renewal/barantas.ca-0002.conf
>   /usr/local/etc/certbot/certs/renewal/fundraiseukraine.ca-0001.conf
>   /usr/local/etc/certbot/certs/renewal/fundraiseukraine.ca-0002.conf
>   /usr/local/etc/certbot/certs/renewal/lawsonpark.ca-0001.conf
>   /usr/local/etc/certbot/certs/renewal/lawsonpark.ca-0002.conf
>   /usr/local/etc/certbot/certs/renewal/mcklpoa.ca-0001.conf
>   /usr/local/etc/certbot/certs/renewal/mcklpoa.ca-0002.conf
>   /usr/local/etc/certbot/certs/renewal/mfno.org-0001.conf
>   /usr/local/etc/certbot/certs/renewal/mfno.org-0002.conf
>   /usr/local/etc/certbot/certs/renewal/ursulak.ca-0001.conf
>   /usr/local/etc/certbot/certs/renewal/ursulak.ca-0002.conf
>   /usr/local/etc/certbot/certs/renewal/ursulak.ca-0003.conf

cursulak · January 12, 2025, 12:29am

So I did run the update on all but the barantas.ca domain.

The new cert for mfno for example is in
/arhcive/mfno.org/cert2.pem and this expires April 2025.
all them except barantas have an april 2025 cert already made.

MikeMcQ · January 12, 2025, 3:15am

Let's focus on one of the domains and this likely leads to the same fix needed for all.

Certificate Name: mfno.org
 Serial Number: 41a9bded0c1291bb23f46ffa70b9c141dca
 Key Type: ECDSA
 Domains: *.mfno.org
 Expiry Date: 2025-01-27 23:19:17+00:00 (VALID: 15 days)
 Certificate Path: /usr/local/etc/certbot/certs/live/mfno.org/fullchain.pem
 Private Key Path: /usr/local/etc/certbot/certs/live/mfno.org/privkey.pem

What do these commands show (omit sudo if you don't need it)
Please show the command followed by its output. Wrap each result in the "Preformatted Text" formatting option if you can.

sudo ls -l /usr/local/etc/certbot/certs/live/mfno.org
sudo ls -l /usr/local/etc/certbot/certs/archive/mfno.org
sudo ls -l /usr/local/etc/certbot/certs/live/mfno.org-0001
sudo ls -l /usr/local/etc/certbot/certs/archive/mfno.org-0001
sudo ls -l /usr/local/etc/certbot/certs/live/mfno.org-0002
sudo ls -l /usr/local/etc/certbot/certs/archive/mfno.org-0002

And, show the contents of these files

/usr/local/etc/certbot/certs/renewal/mfno.org.conf
/usr/local/etc/certbot/certs/renewal/mfno.org-0001.conf
/usr/local/etc/certbot/certs/renewal/mfno.org-0002.conf

And, we should review output of this too. Once we start deleting unused cert profiles this helps ensure we don't break Apache

sudo apachectl -t -D DUMP_VHOSTS

You may need httpd or apache2ctl instead of apachectl. I don't recall which one you o/s uses.

cursulak · January 12, 2025, 11:55am

% ls -l /usr/local/etc/certbot/certs/live/mfno.org
total 8
-rw-r--r-- 1 chris wheel 692 29 Oct 20:17 README
lrwxr-xr-x 1 chris wheel 37 10 Jan 16:53 cert.pem -> ../../archive/mfno.org-0002/cert1.pem
lrwxr-xr-x 1 chris wheel 38 10 Jan 16:53 chain.pem -> ../../archive/mfno.org-0002/chain1.pem
lrwxr-xr-x 1 chris wheel 42 10 Jan 16:53 fullchain.pem -> ../../archive/mfno.org-0002/fullchain1.pem
lrwxr-xr-x 1 chris wheel 40 10 Jan 16:53 privkey.pem -> ../../archive/mfno.org-0002/privkey1.pem

% ls -l /usr/local/etc/certbot/certs/archive/mfno.org
total 208
-rw-r--r--+ 1 chris wheel 1476 12 Oct 2023 cert1.pem
-rw-r--r--+ 1 chris wheel 1261 10 Jan 16:53 cert2.pem
-rw-r--r--+ 1 chris wheel 1476 7 Apr 2024 cert3.pem
-rw-r--r--+ 1 chris wheel 1257 10 Jun 2024 cert4.pem
-rw-r--r-- 1 chris wheel 1261 20 Aug 08:39 cert5.pem
-rw-r--r-- 1 chris wheel 1261 29 Oct 19:53 cert6.pem
-rw-r--r--+ 1 chris wheel 3749 12 Oct 2023 chain1.pem
-rw-r--r--+ 1 chris wheel 1566 10 Jan 16:53 chain2.pem
-rw-r--r--+ 1 chris wheel 1826 7 Apr 2024 chain3.pem
-rw-r--r--+ 1 chris wheel 1566 10 Jun 2024 chain4.pem
-rw-r--r-- 1 chris wheel 1566 20 Aug 08:39 chain5.pem
-rw-r--r-- 1 chris wheel 1566 29 Oct 19:53 chain6.pem
-rw-r--r--+ 1 chris wheel 5225 12 Oct 2023 fullchain1.pem
-rw-r--r--+ 1 chris wheel 2827 10 Jan 16:53 fullchain2.pem
-rw-r--r--+ 1 chris wheel 3302 7 Apr 2024 fullchain3.pem
-rw-r--r--+ 1 chris wheel 2823 10 Jun 2024 fullchain4.pem
-rw-r--r-- 1 chris wheel 2827 20 Aug 08:39 fullchain5.pem
-rw-r--r-- 1 chris wheel 2827 29 Oct 19:53 fullchain6.pem
-rw-r--r--+ 1 chris wheel 241 12 Oct 2023 privkey1.pem
-rw-r--r-- 1 chris wheel 241 10 Jan 16:53 privkey2.pem
-rw-r--r--+ 1 chris wheel 241 4 Jan 2024 privkey2x.pem
-rw-r--r--+ 1 chris wheel 241 7 Apr 2024 privkey3.pem
-rw-r--r--+ 1 chris wheel 241 10 Jun 2024 privkey4.pem
-rw-r--r-- 1 chris wheel 241 20 Aug 08:39 privkey5.pem
-rw-r--r-- 1 chris wheel 241 29 Oct 19:53 privkey6.pem

ls -l /usr/local/etc/certbot/certs/live/mfno.org-0001
ls: /usr/local/etc/certbot/certs/live/mfno.org-0001: No such file or directory
4
% ls -l /usr/local/etc/certbot/certs/archive/mfno.org-0001
total 32
-rw-r--r-- 1 chris wheel 1257 29 Oct 19:35 cert1.pem
-rw-r--r-- 1 chris wheel 1566 29 Oct 19:35 chain1.pem
-rw-r--r-- 1 chris wheel 2823 29 Oct 19:35 fullchain1.pem
-rw------- 1 chris wheel 241 29 Oct 19:35 privkey1.pem

% /usr/local/etc/certbot/certs/live/mfno.org-0002
zsh: no such file or directory: /usr/local/etc/certbot/certs/live/mfno.org-0002

6

ls -l /usr/local/etc/certbot/certs/archive/mfno.org-0002
total 32
-rw-r--r-- 1 chris wheel 1261 29 Oct 20:17 cert1.pem
-rw-r--r-- 1 chris wheel 1566 29 Oct 20:17 chain1.pem
-rw-r--r-- 1 chris wheel 2827 29 Oct 20:17 fullchain1.pem
-rw------- 1 chris wheel 241 29 Oct 20:17 privkey1.pem

7 - Note these were run before upgrading to version 3.01

> /usr/local/etc/certbot/certs/renewal/mfno.org.conf
> -
> # renew_before_expiry = 30 days
> version = 2.11.0
> archive_dir = /usr/local/etc/certbot/certs/archive/mfno.org
> cert = /usr/local/etc/certbot/certs/live/mfno.org/cert.pem
> privkey = /usr/local/etc/certbot/certs/live/mfno.org/privkey.pem
> chain = /usr/local/etc/certbot/certs/live/mfno.org/chain.pem
> fullchain = /usr/local/etc/certbot/certs/live/mfno.org/fullchain.pem
> 
> # Options used in the renewal process
> [renewalparams]
> account = cfd78254ad72729276aba186ada18f68
> config_dir = /usr/local/etc/certbot/certs
> work_dir = /usr/local/etc/certbot
> logs_dir = /usr/local/etc/certbot/logs
> authenticator = manual
> server = https://acme-v02.api.letsencrypt.org/directory
> key_type = ecdsa
> pref_challs = dns-01,

8

> /usr/local/etc/certbot/certs/renewal/mfno.org-0001.conf
> -
> # renew_before_expiry = 30 days
> version = 2.11.0
> archive_dir = /usr/local/etc/certbot/certs/archive/mfno.org-0001
> cert = /usr/local/etc/certbot/certs/live/mfno.org-0001/cert.pem
> privkey = /usr/local/etc/certbot/certs/live/mfno.org-0001/privkey.pem
> chain = /usr/local/etc/certbot/certs/live/mfno.org-0001/chain.pem
> fullchain = /usr/local/etc/certbot/certs/live/mfno.org-0001/fullchain.pem
> 
> # Options used in the renewal process
> [renewalparams]
> account = cfd78254ad72729276aba186ada18f68
> pref_challs = dns-01,
> config_dir = /usr/local/etc/certbot/certs
> work_dir = /usr/local/etc/certbot
> logs_dir = /usr/local/etc/certbot/logs
> authenticator = manual
> server = https://acme-v02.api.letsencrypt.org/directory
> key_type = ecdsa

9

> /usr/local/etc/certbot/certs/renewal/mfno.org-0002.conf
> -
> # renew_before_expiry = 30 days
> version = 2.11.0
> archive_dir = /usr/local/etc/certbot/certs/archive/mfno.org-0002
> cert = /usr/local/etc/certbot/certs/live/mfno.org-0002/cert.pem
> privkey = /usr/local/etc/certbot/certs/live/mfno.org-0002/privkey.pem
> chain = /usr/local/etc/certbot/certs/live/mfno.org-0002/chain.pem
> fullchain = /usr/local/etc/certbot/certs/live/mfno.org-0002/fullchain.pem
> 
> # Options used in the renewal process
> [renewalparams]
> account = cfd78254ad72729276aba186ada18f68
> pref_challs = dns-01,
> config_dir = /usr/local/etc/certbot/certs
> work_dir = /usr/local/etc/certbot
> logs_dir = /usr/local/etc/certbot/logs
> authenticator = manual
> server = https://acme-v02.api.letsencrypt.org/directory
> key_type = ecdsa

10 - vhosts dump, although I can attach actual
% sudo apachectl -t -D DUMP_VHOSTS
Password:
AH00112: Warning: DocumentRoot [/usr/local/var/www/barantas] does not exist
AH00112: Warning: DocumentRoot [/usr/local/var/www/barantas] does not exist
AH00112: Warning: DocumentRoot [/usr/local/var/www/academy] does not exist
AH00112: Warning: DocumentRoot [/usr/local/var/www/academy] does not exist

> VirtualHost configuration:
> *:80                   is a NameVirtualHost
>          default server lawsonpark.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:26)
>          port 80 namevhost lawsonpark.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:26)
>                  alias www.lawsonpark.ca
>                  wild alias *.lawsonpark.ca
>          port 80 namevhost mcklpoa.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:61)
>                  alias www.mcklpoa.ca
>                  wild alias *.mcklpoa.ca
>          port 80 namevhost fundraiseukraine.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:95)
>                  alias www.fundraiseukraine.ca
>                  wild alias *.fundraiseukraine.ca
>          port 80 namevhost mfno.org (/usr/local/etc/httpd/extra/httpd-vhosts.conf:128)
>                  alias www.mfno.org
>                  wild alias *.mfno.org
>          port 80 namevhost ursulak.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:160)
>                  alias www.ursulak.ca
>                  alias chris.ursulak.ca
>                  alias cursulak.ursulak.ca
>          port 80 namevhost nina.ursulak.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:192)
>                  alias nina.ursulak.ca
>          port 80 namevhost barantas.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:224)
>                  alias www.barantas.ca
>          port 80 namevhost academy.barantas.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:256)
> *:443                  is a NameVirtualHost
>          default server lawsonpark.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:40)
>          port 443 namevhost lawsonpark.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:40)
>                  alias www.lawsonpark.ca
>                  wild alias *.lawsonpark.ca
>          port 443 namevhost mcklpoa.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:75)
>                  alias www.mcklpoa.ca
>                  wild alias *.mcklpoa.ca
>          port 443 namevhost fundraiseukraine.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:109)
>                  alias www.fundraiseukraine.ca
>                  wild alias *.fundraiseukraine.ca
>          port 443 namevhost mfno.org (/usr/local/etc/httpd/extra/httpd-vhosts.conf:142)
>                  alias www.mfno.org
>                  wild alias *.mfno.org
>          port 443 namevhost ursulak.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:174)
>                  alias www.ursulak.ca
>                  alias chris.ursulak.ca
>                  alias cursulak.ursulak.ca
>          port 443 namevhost nina.ursulak.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:206)
>                  alias nina.ursulak.ca
>          port 443 namevhost barantas.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:238)
>                  alias www.barantas.ca
>          port 443 namevhost academy.barantas.ca (/usr/local/etc/httpd/extra/httpd-vhosts.conf:269)

http-vhosts.conf
> # Virtual Hosts
> #
> # Required modules: mod_log_config
> 
> # If you want to maintain multiple domains/hostnames on your
> # machine you can setup VirtualHost containers for them. Most configurations
> # use only name-based virtual hosts so the server doesn't need to worry about
> # IP addresses. This is indicated by the asterisks in the directives below.
> #
> # Please see the documentation at 
> # <URL:http://httpd.apache.org/docs/2.4/vhosts/>
> # for further details before you try to setup virtual hosts.
> #
> # You may use the command line option '-S' to verify your virtual host
> # configuration.
> 
> #
> # VirtualHost example:
> # Almost any Apache directive may go into a VirtualHost container.
> # The first VirtualHost section is used for all requests that do not
> # match a ServerName or ServerAlias in any <VirtualHost> block.
> #
> 
> 
> #----LAWSONPARK.ca------------
> <VirtualHost *:80>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/lawsonpark"
>     ServerName lawsonpark.ca
>     ServerAlias www.lawsonpark.ca *.lawsonpark.ca
>     ErrorLog "/usr/local/var/www/log/lawsonspark.ca-error_log"
>     CustomLog "/usr/local/var/www/log/lawsonpark.ca-access_log" common
>         <Directory "/usr/local/var/www/lawsonpark">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
> </VirtualHost>
> 
> <VirtualHost *:443>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/lawsonpark"
>     ServerName lawsonpark.ca
>     ServerAlias www.lawsonpark.ca *.lawsonpark.ca
>     ErrorLog "/usr/local/var/www/log/lawsonspark.ca-error_log"
>     CustomLog "/usr/local/var/www/log/lawsonpark.ca-access_log" common
>         <Directory "/usr/local/var/www/lawsonpark">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
> 	SSLEngine on
>   	SSLCertificateFile /usr/local/etc/certbot/certs/live/lawsonpark.ca/fullchain.pem
> 	SSLCertificateKeyFile /usr/local/etc/certbot/certs/live/lawsonpark.ca/privkey.pem
> </VirtualHost>
> 
> 
> 
> 
> #----MCKLPOA.ca------------
> <VirtualHost *:80>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/mcklpoa"
>     ServerName mcklpoa.ca
>     ServerAlias www.mcklpoa.ca *.mcklpoa.ca
>     ErrorLog "/usr/local/var/www/log/mcklpoa.ca-error_log"
>     CustomLog "/usr/local/var/www/log/mcklpoa.ca-access_log" common
>         <Directory "/usr/local/var/www/mcklpoa">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
> </VirtualHost>
> 
> <VirtualHost *:443>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/mcklpoa"
>     ServerName mcklpoa.ca
>     ServerAlias www.mcklpoa.ca *.mcklpoa.ca
>     ErrorLog "/usr/local/var/www/log/mcklpoa.ca-error_log"
>     CustomLog "/usr/local/var/www/log/mcklpoa.ca-access_log" common
>         <Directory "/usr/local/var/www/mcklpoa">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
>     SSLEngine on
>   	SSLCertificateFile /usr/local/etc/certbot/certs/live/mcklpoa.ca/fullchain.pem
> 	SSLCertificateKeyFile /usr/local/etc/certbot/certs/live/mcklpoa.ca/privkey.pem
> </VirtualHost>
> 
> 
> 
> #----FUNDRAISEUKRAINEca------------
> <VirtualHost *:80>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/fundraiseukraine"
>     ServerName fundraiseukraine.ca
>     ServerAlias www.fundraiseukraine.ca *.fundraiseukraine.ca
>     ErrorLog "/usr/local/var/www/log/fundraiseukraine.ca-error_log"
>     CustomLog "/usr/local/var/www/log/fundraiseukraine.ca-access_log" common
>         <Directory "/usr/local/var/www/fundraiseukraine">
>             Options Indexes FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
> </VirtualHost>
> 
>  <VirtualHost *:443>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/fundraiseukraine"
>     ServerName fundraiseukraine.ca
>     ServerAlias www.fundraiseukraine.ca *.fundraiseukraine.ca
>     ErrorLog "/usr/local/var/www/log/fundraiseukraine.ca-error_log"
>     CustomLog "/usr/local/var/www/log/fundraiseukraine.ca-access_log" common
>         <Directory "/usr/local/var/www/fundraiseukraine">
>             Options Indexes FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
>     SSLEngine on
>   	SSLCertificateFile /usr/local/etc/certbot/certs/live/fundraiseukraine.ca/fullchain.pem
> 	SSLCertificateKeyFile /usr/local/etc/certbot/certs/live/fundraiseukraine.ca/privkey.pem
> </VirtualHost>
> 
> 
> #----MFNO.org------------
> <VirtualHost *:80>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/mfno"
>     ServerName mfno.org
>     ServerAlias www.mfno.org *.mfno.org
>     ErrorLog "/usr/local/var/www/log/mfno.org-error_log"
>     CustomLog "/usr/local/var/www/log/mfno.org-access_log" common
>         <Directory "/usr/local/var/www/mfno">
>             Options Indexes FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
> </VirtualHost>
> 
>  <VirtualHost *:443>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/mfno"
>     ServerName mfno.org
>     ServerAlias www.mfno.org *.mfno.org
>     ErrorLog "/usr/local/var/www/log/mfno.org-error_log"
>     CustomLog "/usr/local/var/www/log/mfno.org-access_log" common
>         <Directory "/usr/local/var/www/mfno">
>             Options Indexes FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
>     SSLEngine on
>   	SSLCertificateFile /usr/local/etc/certbot/certs/live/mfno.org/fullchain.pem
> 	SSLCertificateKeyFile /usr/local/etc/certbot/certs/live/mfno.org/privkey.pem
> </VirtualHost>
> 
> #----URUSULAK.ca------------
> <VirtualHost *:80>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/ursulak"
>     ServerName ursulak.ca
>     ServerAlias www.ursulak.ca chris.ursulak.ca cursulak.ursulak.ca
>     ErrorLog "/usr/local/var/www/log/ursulak.ca-error_log"
>     CustomLog "/usr/local/var/www/log/ursulak.ca-access_log" common
>         <Directory "/usr/local/var/www/ursulak">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
> </VirtualHost>
> 
> <VirtualHost *:443>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/ursulak"
>     ServerName ursulak.ca
>     ServerAlias www.ursulak.ca chris.ursulak.ca cursulak.ursulak.ca
>     ErrorLog "/usr/local/var/www/log/ursulak.ca-error_log"
>     CustomLog "/usr/local/var/www/log/ursulak.ca-access_log" common
>         <Directory "/usr/local/var/www/ursulak">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
>     SSLEngine on
>   	SSLCertificateFile /usr/local/etc/certbot/certs/live/ursulak.ca/fullchain.pem
> 	SSLCertificateKeyFile /usr/local/etc/certbot/certs/live/ursulak.ca/privkey.pem
> </VirtualHost>
> 
> #----NINA.URUSULAK.ca------------
> <VirtualHost *:80>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/nina"
>     ServerName nina.ursulak.ca
>     ServerAlias nina.ursulak.ca
>     ErrorLog "/usr/local/var/www/log/nina.ca-error_log"
>     CustomLog "/usr/local/var/www/log/nina.ca-access_log" common
>         <Directory "/usr/local/var/www/nina">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
> </VirtualHost>
> 
> <VirtualHost *:443>
>     ServerAdmin cursulak@gmail.com
>     DocumentRoot "/usr/local/var/www/nina"
>     ServerName nina.ursulak.ca
>     ServerAlias nina.ursulak.ca
>     ErrorLog "/usr/local/var/www/log/nina.ca-error_log"
>     CustomLog "/usr/local/var/www/log/nina.ca-access_log" common
>         <Directory "/usr/local/var/www/nina">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
>     SSLEngine on
>   	SSLCertificateFile /usr/local/etc/certbot/certs/live/ursulak.ca/fullchain.pem
> 	SSLCertificateKeyFile /usr/local/etc/certbot/certs/live/ursulak.ca/privkey.pem
> </VirtualHost>
> 
> #----BARANTAS.ca------------
> <VirtualHost *:80>
>     ServerAdmin chris@barantas.ca
>     DocumentRoot "/usr/local/var/www/barantas"
>     ServerName barantas.ca
>     ServerAlias www.barantas.ca
>     ErrorLog "/usr/local/var/www/log/barantas.ca-error_log"
>     CustomLog "/usr/local/var/www/log/barantas.ca-access_log" common
>         <Directory "/usr/local/var/www/barantas">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
> </VirtualHost>
> 
> <VirtualHost *:443>
>     ServerAdmin chris@barantas.ca
>     DocumentRoot "/usr/local/var/www/barantas"
>     ServerName barantas.ca
>     ServerAlias www.barantas.ca
>     ErrorLog "/usr/local/var/www/log/barantas.ca-error_log"
>     CustomLog "/usr/local/var/www/log/barantas.ca-access_log" common
>         <Directory "/usr/local/var/www/barantas">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
>     SSLEngine on
>   	SSLCertificateFile /usr/local/etc/certbot/certs/live/barantas.ca/fullchain.pem
> 	SSLCertificateKeyFile /usr/local/etc/certbot/certs/live/barantas.ca/privkey.pem
> </VirtualHost>
> 
> #----ACADEMY.BARANTAS.ca------------
> <VirtualHost *:80>
>     ServerAdmin chris@barantas.ca
>     DocumentRoot "/usr/local/var/www/academy"
>     ServerName academy.barantas.ca
>     ErrorLog "/usr/local/var/www/log/academy.barantas.ca-error_log"
>     CustomLog "/usr/local/var/www/log/academy.barantas.ca-access_log" common
>         <Directory "/usr/local/var/www/academy">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
> </VirtualHost>
> 
> <VirtualHost *:443>
>     ServerAdmin chris@barantas.ca
>     DocumentRoot "/usr/local/var/www/academy"
>     ServerName academy.barantas.ca
>     ErrorLog "/usr/local/var/www/log/academy.barantas.ca-error_log"
>     CustomLog "/usr/local/var/www/log/academy.barantas.ca-access_log" common
>         <Directory "/usr/local/var/www/academy">
>             Options Indexes MultiViews FollowSymLinks
>             AllowOverride All
>             Require all granted
>     </Directory>
>     SSLEngine on
>   	SSLCertificateFile /usr/local/etc/certbot/certs/live/barantas.ca/fullchain.pem
> 	SSLCertificateKeyFile /usr/local/etc/certbot/certs/live/barantas.ca/privkey.pem
> </VirtualHost>

MikeMcQ · January 12, 2025, 3:24pm

Well, that is an interesting, um, mess I don't think it fruitful to understand how it got to this state. And, there are a few ways to get this corrected. But, let's try this

Delete these from "base" mfno.org archive folder. After these deletes only the very old 1 set and the latest 2 set remain (check w/ls -l after)

rm /usr/local/etc/certbot/certs/archive/mfno.org/privkey2x.pem
rm /usr/local/etc/certbot/certs/archive/mfno.org/*3.pem
rm /usr/local/etc/certbot/certs/archive/mfno.org/*4.pem
rm /usr/local/etc/certbot/certs/archive/mfno.org/*5.pem
rm /usr/local/etc/certbot/certs/archive/mfno.org/*6.pem

Fix the symlink to point live to the 2 set in archive. You will need to delete the existing symlinks first

cd /usr/local/etc/certbot/certs/live/mfno.org
# make sure in correct folder then:
sudo rm *.pem
sudo ln -s ../../archive/mfno.org/cert2.pem cert.pem
sudo ln -s ../../archive/mfno.org/chain2.pem chain.pem
sudo ln -s ../../archive/mfno.org/fullchain2.pem fullchain.pem
sudo ln -s ../../archive/mfno.org/privkey2.pem privkey.pem

Then, show output of below. Please do not yet run any other Certbot commands. If I missed a step it is easier to fix before we do that.

certbot certificates --cert-name mfno.org
ls -l /usr/local/etc/certbot/certs/live/mfno.org

cursulak · January 13, 2025, 2:09pm

Here we go. Only periodically at the computer as dealing with another issue, my apologies.

chris@Ursulak mfno.org % ls -l
total 8
-rw-r--r--+ 1 chris wheel 692 29 Oct 20:17 README
lrwxr-xr-x+ 1 chris wheel 32 13 Jan 09:01 cert.pem -> ../../archive/mfno.org/cert2.pem
lrwxr-xr-x+ 1 chris wheel 33 13 Jan 09:01 chain.pem -> ../../archive/mfno.org/chain2.pem
lrwxr-xr-x+ 1 chris wheel 37 13 Jan 09:02 fullchain.pem -> ../../archive/mfno.org/fullchain2.pem
lrwxr-xr-x+ 1 chris wheel 35 13 Jan 09:02 privkey.pem -> ../../archive/mfno.org/privkey2.pem

and then when checking where it's pointing to, got strange log permissions issues. Never had this before so went and fixed that. Then running again...

Why is the behavior despite asking only for mfno output, it reports on all the others as well....?

% certbot certificates --cert-name mfno.org
Saving debug log to /usr/local/etc/certbot/logs/letsencrypt.log
Renewal configuration file /usr/local/etc/certbot/certs/renewal/barantas.ca-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/barantas.ca-0001/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/barantas.ca-0002.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/barantas.ca-0002/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/fundraiseukraine.ca-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/fundraiseukraine.ca-0001/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/fundraiseukraine.ca-0002.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/fundraiseukraine.ca-0002/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/lawsonpark.ca-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/lawsonpark.ca-0001/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/lawsonpark.ca-0002.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/lawsonpark.ca-0002/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/mcklpoa.ca-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/mcklpoa.ca-0001/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/mcklpoa.ca-0002.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/mcklpoa.ca-0002/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/mfno.org-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/mfno.org-0001/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/mfno.org-0002.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/mfno.org-0002/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/ursulak.ca-0001.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/ursulak.ca-0001/cert.pem to be a symlink. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/ursulak.ca-0002.conf produced an unexpected error: renewal config file {} is missing a required file reference. Skipping.
Renewal configuration file /usr/local/etc/certbot/certs/renewal/ursulak.ca-0003.conf produced an unexpected error: expected /usr/local/etc/certbot/certs/live/ursulak.ca-0003/cert.pem to be a symlink. Skipping.

Found the following matching certs:
Certificate Name: mfno.org
Serial Number: 3443143b0990a1992dd80887efbdaa6eaf4
Key Type: ECDSA
Domains: *.mfno.org
Expiry Date: 2025-04-10 20:55:11+00:00 (VALID: 87 days)
Certificate Path: /usr/local/etc/certbot/certs/live/mfno.org/fullchain.pem
Private Key Path: /usr/local/etc/certbot/certs/live/mfno.org/privkey.pem

The following renewal configurations were invalid:
/usr/local/etc/certbot/certs/renewal/barantas.ca-0001.conf
/usr/local/etc/certbot/certs/renewal/barantas.ca-0002.conf
/usr/local/etc/certbot/certs/renewal/fundraiseukraine.ca-0001.conf
/usr/local/etc/certbot/certs/renewal/fundraiseukraine.ca-0002.conf
/usr/local/etc/certbot/certs/renewal/lawsonpark.ca-0001.conf
/usr/local/etc/certbot/certs/renewal/lawsonpark.ca-0002.conf
/usr/local/etc/certbot/certs/renewal/mcklpoa.ca-0001.conf
/usr/local/etc/certbot/certs/renewal/mcklpoa.ca-0002.conf
/usr/local/etc/certbot/certs/renewal/mfno.org-0001.conf
/usr/local/etc/certbot/certs/renewal/mfno.org-0002.conf
/usr/local/etc/certbot/certs/renewal/ursulak.ca-0001.conf
/usr/local/etc/certbot/certs/renewal/ursulak.ca-0002.conf
/usr/local/etc/certbot/certs/renewal/ursulak.ca-0003.conf

cursulak · January 13, 2025, 2:11pm

Generally I do all the certs at once and restart the system to pick up the new certs. I haven't done that as you asked to hold here, but MFNO still showing the january one as cached in apache.

MikeMcQ · January 13, 2025, 2:20pm

I am not sure. Other than Certbot is probably reporting errors as it searches its profiles for the --cert-name requested. It is distracting, I agree.

You could gracefully reload Apache now and you should see mfno using the cert issued in Jan.

But, before we continue I'd like to know if a cert with only the wildcard name *.mfno.org is intended. Because usually the apex name is also included as the wildcard name only covers URL requests for subdomains of the apex. I'd like to walk through updating that carefully given your history of odd path names developing. And, changing the set of domain names on a cert can do that if not done properly.

Put another way, currently requests to https://mfno.org will fail due to invalid cert. Do you want to fix that?

MikeMcQ · January 13, 2025, 2:23pm

Oh, and if Certbot sets up the symlinks the permissions would be:

lrwxrwxrwx

You should probably chmod the ones we made manually just to be safe.

cursulak · January 13, 2025, 2:34pm

The apache virtual hosts repoints mono.org to www.mfno.org. But one of the reasons for the wild card is we run a database in a sub domain that's not publicly available but we use the cert for that as well.

Is it a problem to have apache redirecting to www for all the web traffic? you're proposing a *.mfno.org and www.mfno.org in the same cert?

cursulak · January 13, 2025, 2:36pm

autocorrect changed that, but if you enter just mfno.org you'll find you get to the right place without errors.

MikeMcQ · January 13, 2025, 2:51pm

No. I am proposing mfno.org and *.mfno.org in the same cert.

I am not sure that every browser will ignore the incorrect cert for your apex. If you want to be sure it always works there should be both. You named mfno.org explicitly in your VirtualHost for port 443 so it appears you want it to work.

curl -i https://mfno.org
curl: (60) SSL: no alternative certificate subject name matches target host name 'mfno.org'

Also try mfno.org and www.mfno.org in: https://decoder.link/sslchecker

cursulak · January 13, 2025, 2:59pm

Ok, I'm fine with that to have both, gets to the same point regards, I had just used redirection incorrectly or otherwise it was working. We can do both then. I was hoping to use autorenew so I wouldn't have to play with these very few months, but I understand from the readings since I have wildcard I can't do that.

cursulak · January 13, 2025, 3:01pm

I see in the errors reported above the renewal for mfno is still producing errors. There are three files there mfno.org.conf, mfno.org-0001.conf, mfno.org-0002.conf. 1 and 2 expected sim links but not. Should I remove them? leaving just first?

MikeMcQ · January 13, 2025, 3:09pm

You have several different cert renewal profiles for mfno. I want to get the primary mfno profile working properly first. We will cleanup those other profiles later.

Next, lets adjust your cert to include apex with wildcard. Run this

certbot certonly --manual --preferred-challenges dns --cert-name mfno.org -d mfno.org -d "*.mfno.org"

Certbot may prompt about "expanding" the cert or something like that. Choose Yes.

When done show:

ls -l /usr/local/etc/certbot/certs/{live,archive}/mfno.org

MikeMcQ · January 13, 2025, 3:14pm

It is possible to automate DNS Challenge. But requires an API or way to automatically add / delete the TXT records in your DNS server. It also requires your ACME Client (Certbot in your case) to support the method your DNS provider needs. We can come back to that later.

Topic		Replies	Views
Certbot does not change symlinks Help	5	1324	April 1, 2020
"certbot --apache" only renews first certificate in list after pressing Enter Help	9	745	January 30, 2022
Certbot Updates - Not Able to Renew Cert Help	37	4757	May 31, 2021
Renew worked, but web server not serving new cert Help	10	811	September 6, 2018
Apachectl -S says fullchain.pem does not exist or is empty but certbot says it has been saved Help	14	6238	July 4, 2018

Renewed certificates, but apache and the live link cert is not the current one

Related topics