Certbot unable to renew: Invalid response

Help
1

Hello everyone :slight_smile:

I have been using certbot and letsencrypt for many years now and haven't had any issues - until today :smiley: .Recently I received an email from LE telling me at least 2 domain certificates are going to expire. So today I looked onto the server to find out what's up.

In those first attempts I just executed "certbot certonly --apache" (also with --dry-run later) but the result kept staying the same (see below). After that I checked the install page and found something new is recommended (snapd) which is why I thought maybe something changed and I'm running an old certbot. In any way followed the instructions on Certbot Instructions | Certbot to remove the previous certbot installation and use snapd from here on but to my surprise the result kept staying the same.

However, I then wanted to know for each single domain what's up so I ran the certbot update 4 times to go through each of the 4 domains/websites. To my absolute surprise the ones that didn't want to cert before now did cert - but there's still 1 domain/website that receives the error I have been seeing since my first attempts today. You'll find the full log below.

My domain is (the 1 left from 4 domains):
forum.survivetheforest.net

I ran this command:
certbot certonly --apache

It produced this output (full log of the 4 per-domain attempts):

root@server:/var/www# certbot certonly --apache
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Which names would you like to activate HTTPS for?
We recommend selecting either all domains, or all domains in a VirtualHost/server block.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: forum.stranded-games.net
2: forum.survivetheforest.net
3: modapi.survivetheforest.net
4: www.tmnttoys.net
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 1

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An RSA certificate named forum.stranded-games.net already exists. Do you want to
update its key type to ECDSA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate key type/(K)eep existing key type: U
Renewing an existing certificate for forum.stranded-games.net

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/forum.stranded-games.net/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/forum.stranded-games.net/privkey.pem
This certificate expires on 2023-06-07.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
root@server:/var/www# certbot certonly --apache
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Which names would you like to activate HTTPS for?
We recommend selecting either all domains, or all domains in a VirtualHost/server block.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: forum.stranded-games.net
2: forum.survivetheforest.net
3: modapi.survivetheforest.net
4: www.tmnttoys.net
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 2

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An RSA certificate named forum.survivetheforest.net already exists. Do you want
to update its key type to ECDSA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate key type/(K)eep existing key type: U
Renewing an existing certificate for forum.survivetheforest.net

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
  Domain: forum.survivetheforest.net
  Type:   unauthorized
  Detail: 109.200.202.106: Invalid response from http://forum.survivetheforest.net/.well-known/acme-challenge/0HpGdSC62I4PA39ZpAnJqdn5F4sjf_uq_Q1V15pIuuI: 404

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
root@server:/var/www# certbot certonly --apache
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Which names would you like to activate HTTPS for?
We recommend selecting either all domains, or all domains in a VirtualHost/server block.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: forum.stranded-games.net
2: forum.survivetheforest.net
3: modapi.survivetheforest.net
4: www.tmnttoys.net
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 3

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An RSA certificate named modapi.survivetheforest.net already exists. Do you want
to update its key type to ECDSA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate key type/(K)eep existing key type: U
Renewing an existing certificate for modapi.survivetheforest.net

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/modapi.survivetheforest.net/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/modapi.survivetheforest.net/privkey.pem
This certificate expires on 2023-06-07.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
root@server:/var/www# certbot certonly --apache
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Which names would you like to activate HTTPS for?
We recommend selecting either all domains, or all domains in a VirtualHost/server block.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: forum.stranded-games.net
2: forum.survivetheforest.net
3: modapi.survivetheforest.net
4: www.tmnttoys.net
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 4

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An RSA certificate named www.tmnttoys.net already exists. Do you want to update
its key type to ECDSA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate key type/(K)eep existing key type: U
Renewing an existing certificate for www.tmnttoys.net

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/www.tmnttoys.net/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/www.tmnttoys.net/privkey.pem
This certificate expires on 2023-06-07.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
root@server:/var/www# certbot certonly --apache
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Which names would you like to activate HTTPS for?
We recommend selecting either all domains, or all domains in a VirtualHost/server block.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: forum.stranded-games.net
2: forum.survivetheforest.net
3: modapi.survivetheforest.net
4: www.tmnttoys.net
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 2

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An RSA certificate named forum.survivetheforest.net already exists. Do you want
to update its key type to ECDSA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate key type/(K)eep existing key type: U
Renewing an existing certificate for forum.survivetheforest.net
An unexpected error occurred:
Error creating new order :: too many failed authorizations recently: see https://letsencrypt.org/docs/failed-validation-limit/
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
root@server:/var/www#

My web server is (include version):
Apache/2.4.25 (Debian)

The operating system my web server runs on is (include version):
Debian GNU/Linux 9.13 (stretch)

My hosting provider, if applicable, is:
i3d

I can login to a root shell on my machine (yes or no, or I don't know):
yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 2.4.0

As a final note to this post at this point: I did try to find this .well-known folder on my system but it doesn't seem to be there at all (hence the 404) but I don't understand why certbot expects this folder and how I am supposed to create it with its corresponding contents. I checked with the 2 websites for error-analysis whereas the 1st one (letsdebug) didn't show any issues, but the 2nd one (check-your-website) responded with the error I and in detail also was showing under this I mark the 404 for the .well-known thing.

I am thankful for every help and interested to turn a bit wiser regarding certbot usage.
Thank you up front, Chris2k

1 Like
2

Welcome to the community @Chris2k

Certbot creates that folder and places the contents in it as needed and removes after.

Usually a 404 error when using the Apache authenticator is a problem with the Apache config. Can you show output of this:

apachectl -t -D DUMP_VHOSTS

And, just fyi, you can do this to test just one cert renewal

certbot renew --dry-run --cert-name forum.survivetheforest.net
4 Likes
3 
root@server:/var/www# apachectl -t -D DUMP_VHOSTS
VirtualHost configuration:
*:80                   is a NameVirtualHost
         default server xxxx.i3d.net (/etc/apache2/sites-enabled/000-default.conf:1)
         port 80 namevhost xxxx.i3d.net (/etc/apache2/sites-enabled/000-default.conf:1)
         port 80 namevhost forum.survivetheforest.net (/etc/apache2/sites-enabled/000-default.conf:32)
         port 80 namevhost forum.stranded-games.net (/etc/apache2/sites-enabled/000-default.conf:68)
         port 80 namevhost modapi.survivetheforest.net (/etc/apache2/sites-enabled/000-default.conf:104)
*:443                  is a NameVirtualHost
         default server xxxx.i3d.net (/etc/apache2/sites-enabled/000-default.conf:15)
         port 443 namevhost xxxx.i3d.net (/etc/apache2/sites-enabled/000-default.conf:15)
         port 443 namevhost forum.survivetheforest.net (/etc/apache2/sites-enabled/000-default.conf:48)
         port 443 namevhost forum.stranded-games.net (/etc/apache2/sites-enabled/000-default.conf:84)
         port 443 namevhost modapi.survivetheforest.net (/etc/apache2/sites-enabled/000-default.conf:120)
         port 443 namevhost www.tmnttoys.net (/etc/apache2/sites-enabled/000-default.conf:157)
root@server:/var/www#

root@server:/var/www# certbot renew --dry-run --cert-name forum.survivetheforest.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/forum.survivetheforest.net.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Simulating renewal of an existing certificate for forum.survivetheforest.net
Failed to renew certificate forum.survivetheforest.net with error: Could not bind TCP port 80 because it is already in use by another process on this system (such as a web server). Please stop the program in question and then try again.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All simulated renewals failed. The following certificates could not be renewed:
  /etc/letsencrypt/live/forum.survivetheforest.net/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
root@server:/var/www#
4

Huh. Well that's different than the error shown in your first post.

What are contents of this

/etc/letsencrypt/renewal/forum.survivetheforest.net.conf
3 Likes
5 
root@server:/var/www# cat /etc/letsencrypt/renewal/forum.survivetheforest.net.conf
# renew_before_expiry = 30 days
version = 0.28.0
archive_dir = /etc/letsencrypt/archive/forum.survivetheforest.net
cert = /etc/letsencrypt/live/forum.survivetheforest.net/cert.pem
privkey = /etc/letsencrypt/live/forum.survivetheforest.net/privkey.pem
chain = /etc/letsencrypt/live/forum.survivetheforest.net/chain.pem
fullchain = /etc/letsencrypt/live/forum.survivetheforest.net/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = standalone
post_hook = apache2ctl -k start
account = xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
pre_hook = apache2ctl -k stop
server = https://acme-v02.api.letsencrypt.org/directory
root@server:/var/www#
6

Is there a reason you need to use the standalone authenticator? And, are your /renewal/ conf files for the other domains also using standalone?

I am inclined to have you re-make the cert for forum with apache authenticator and fixing the reason for the 404 error in your first post using that.

But, if you've already gone down that path and failed no sense trying again

4 Likes
7

To be honest I don't even know what that standalone option means. Since LE and certbot never gave me issues before and pretty much were always operating on its own I never bothered any further.

Oh wow, the other /renewal/ files look different. Saying different versioin and "apache" instead of "standalone".

root@server:/var/www# cat /etc/letsencrypt/renewal/modapi.survivetheforest.net.conf
# renew_before_expiry = 30 days
version = 2.4.0
archive_dir = /etc/letsencrypt/archive/modapi.survivetheforest.net
cert = /etc/letsencrypt/live/modapi.survivetheforest.net/cert.pem
privkey = /etc/letsencrypt/live/modapi.survivetheforest.net/privkey.pem
chain = /etc/letsencrypt/live/modapi.survivetheforest.net/chain.pem
fullchain = /etc/letsencrypt/live/modapi.survivetheforest.net/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = apache
account = xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
server = https://acme-v02.api.letsencrypt.org/directory
installer = apache
key_type = ecdsa

root@server:/var/www# cat /etc/letsencrypt/renewal/forum.stranded-games.net.conf
# renew_before_expiry = 30 days
version = 2.4.0
archive_dir = /etc/letsencrypt/archive/forum.stranded-games.net
cert = /etc/letsencrypt/live/forum.stranded-games.net/cert.pem
privkey = /etc/letsencrypt/live/forum.stranded-games.net/privkey.pem
chain = /etc/letsencrypt/live/forum.stranded-games.net/chain.pem
fullchain = /etc/letsencrypt/live/forum.stranded-games.net/fullchain.pem

# Options used in the renewal process
[renewalparams]
installer = apache
account = xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
server = https://acme-v02.api.letsencrypt.org/directory
authenticator = apache
key_type = ecdsa

root@server:/var/www# cat /etc/letsencrypt/renewal/www.tmnttoys.net.conf
# renew_before_expiry = 30 days
version = 2.4.0
archive_dir = /etc/letsencrypt/archive/www.tmnttoys.net
cert = /etc/letsencrypt/live/www.tmnttoys.net/cert.pem
privkey = /etc/letsencrypt/live/www.tmnttoys.net/privkey.pem
chain = /etc/letsencrypt/live/www.tmnttoys.net/chain.pem
fullchain = /etc/letsencrypt/live/www.tmnttoys.net/fullchain.pem

# Options used in the renewal process
[renewalparams]
installer = apache
account = xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
authenticator = apache
server = https://acme-v02.api.letsencrypt.org/directory
key_type = ecdsa
root@server:/var/www#
8

Can you show the VirtualHost for forum? Either show the entire 000-default.conf or just the VirtualHost starting at line 32.

And, fyi, as for your others, they use apache now as you re-ran the initial cert request process in post #1. Usually once you have the cert you just run certbot renew and this renew is usually done in cron or a systemd timer. For some reason your forum domain didn't work though.

4 Likes
9 
root@server:/var/www# cat /etc/apache2/sites-enabled/000-default.conf
<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www
        <Directory /var/www/>
                Options FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
        ServerSignature Off
</VirtualHost>

<VirtualHost *:443>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www
        SSLEngine on
        <Directory /var/www/>
                Options FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        ErrorLog ${APACHE_LOG_DIR}/error_ssl.log
        CustomLog ${APACHE_LOG_DIR}/access_ssl.log combined
        ServerSignature Off
        SSLCertificateFile /etc/letsencrypt/live/forum.survivetheforest.net/fullchain.pem
        SSLCertificateKeyFile /etc/letsencrypt/live/forum.survivetheforest.net/privkey.pem
</VirtualHost>

<VirtualHost *:80>
        ServerAdmin info@survivetheforest.net
        ServerName forum.survivetheforest.net
        DocumentRoot /var/www/theforest/
        <Directory /var/www/theforest/>
                Options FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>
        CustomLog /var/log/apache2/forum-theforest-net/access.log common
        LogLevel info
        ErrorLog /var/log/apache2/forum-theforest-net/error.log
        ServerSignature Off
</VirtualHost>

<VirtualHost *:443>
        ServerAdmin info@survivetheforest.net
        ServerName forum.survivetheforest.net
        DocumentRoot /var/www/theforest/
        SSLEngine on
        <Directory /var/www/theforest/>
                Options FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>
        CustomLog /var/log/apache2/forum-theforest-net/access.log common
        LogLevel info
        ErrorLog /var/log/apache2/forum-theforest-net/error.log
        ServerSignature Off
        SSLCertificateFile /etc/letsencrypt/live/forum.survivetheforest.net/fullchain.pem
        SSLCertificateKeyFile /etc/letsencrypt/live/forum.survivetheforest.net/privkey.pem
        Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>

<VirtualHost *:80>
        ServerAdmin info@stranded-games.net
        ServerName forum.stranded-games.net
        DocumentRoot /var/www/strandedgames/
        <Directory /var/www/strandedgames/>
                Options FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        ErrorLog /var/log/apache2/forum-strandedgames-net/error.log
        LogLevel info
        CustomLog /var/log/apache2/forum-strandedgames-net/access.log common
        ServerSignature Off
</VirtualHost>

<VirtualHost *:443>
        ServerAdmin info@stranded-games.net
        ServerName forum.stranded-games.net
        DocumentRoot /var/www/strandedgames/
        SSLEngine on
        <Directory /var/www/strandedgames/>
                Options FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        ErrorLog /var/log/apache2/forum-strandedgames-net/error.log
        LogLevel info
        CustomLog /var/log/apache2/forum-strandedgames-net/access.log common
        ServerSignature Off
        SSLCertificateFile /etc/letsencrypt/live/forum.stranded-games.net/fullchain.pem
        SSLCertificateKeyFile /etc/letsencrypt/live/forum.stranded-games.net/privkey.pem
        Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>

<VirtualHost *:80>
        ServerAdmin info@survivetheforest.net
        ServerName modapi.survivetheforest.net
        DocumentRoot /var/www/tf_modapi/www/
        <Directory /var/www/tf_modapi/www/>
                Options FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>
        CustomLog /var/log/apache2/tf_modapi/access.log common
        LogLevel info
        ErrorLog /var/log/apache2/tf_modapi/error.log
        ServerSignature Off
</VirtualHost>

<VirtualHost *:443>
        ServerAdmin info@survivetheforest.net
        ServerName modapi.survivetheforest.net
        DocumentRoot /var/www/tf_modapi/www/
        SSLEngine on
        <Directory /var/www/tf_modapi/www/>
                Options FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>
        CustomLog /var/log/apache2/tf_modapi/access.log common
        LogLevel info
        ErrorLog /var/log/apache2/tf_modapi/error.log
        ServerSignature Off
        RewriteEngine on
        SSLCertificateFile /etc/letsencrypt/live/modapi.survivetheforest.net/fullchain.pem
        SSLCertificateKeyFile /etc/letsencrypt/live/modapi.survivetheforest.net/privkey.pem
        Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>

#<VirtualHost *:80>
#       ServerAdmin info@tmnttoys.net
#       ServerName www.tmnttoys.net
#       DocumentRoot /var/www/tmnttoys/
#       <Directory /var/www/tmnttoys/>
#               Options FollowSymLinks MultiViews
#               AllowOverride All
#               Order allow,deny
#               allow from all
#       </Directory>
#       CustomLog /var/log/apache2/tmnttoys-net/access.log common
#       LogLevel info
#       ErrorLog /var/log/apache2/tmnttoys-net/error.log
#       ServerSignature Off
#</VirtualHost>

<VirtualHost *:443>
        ServerAdmin info@tmnttoys.net
        ServerName www.tmnttoys.net
        DocumentRoot /var/www/tmnttoys/
        SSLEngine on
        <Directory /var/www/tmnttoys/>
                Options FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>
        CustomLog /var/log/apache2/tmnttoys-net/access.log common
        LogLevel info
        ErrorLog /var/log/apache2/tmnttoys-net/error.log
        ServerSignature Off
        RewriteEngine on
        SSLCertificateFile /etc/letsencrypt/live/www.tmnttoys.net/fullchain.pem
        SSLCertificateKeyFile /etc/letsencrypt/live/www.tmnttoys.net/privkey.pem
        Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
root@server:/var/www#

So when I change the /renewal/ conf for forum so it looks like one of the others would that work? Or are those changes written into the /renewal/ conf files by another way?

1 Like
10

It is best to let Certbot update the renewal conf files. It will do that after each successful run.

I don't see anything odd in your Apache config. Can you try this

certbot certonly --dry-run --webroot -w /var/www/theforest -d forum.survivetheforest.net
3 Likes
11 
root@server:/var/www# certbot certonly --dry-run --webroot -w /var/www/theforest -d forum.survivetheforest.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An RSA certificate named forum.survivetheforest.net already exists. Do you want
to update its key type to ECDSA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate key type/(K)eep existing key type: U
Simulating renewal of an existing certificate for forum.survivetheforest.net
The dry run was successful.
root@server:/var/www#

After that I looked up contents of /etc/letsencrypt/renewal/forum.survivetheforest.net.conf but it's still the same with standalone, etc

1 Like
12

Yes, --dry-run is just testing. Renewal conf only updates with production certs

What about this now?

certbot certonly --dry-run --apache -d forum.survivetheforest.net
3 Likes
13

Ah yeah, right, it was only a dry-run that doesn't affect anything, where's my head been xD

Here's the result for certbot certonly --dry-run --apache -d forum.survivetheforest.net

root@server:/var/www# certbot certonly --dry-run --apache -d forum.survivetheforest.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An RSA certificate named forum.survivetheforest.net already exists. Do you want
to update its key type to ECDSA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate key type/(K)eep existing key type: U
Simulating renewal of an existing certificate for forum.survivetheforest.net

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
  Domain: forum.survivetheforest.net
  Type:   unauthorized
  Detail: 109.200.202.106: Invalid response from http://forum.survivetheforest.net/.well-known/acme-challenge/6nKQJ7iTWdxoY20wjMCZ6jFy0rtwbn6ibSVcE8uXNsU: 404

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
root@server:/var/www#
1 Like
14

Well, I don't see why --apache fails with that one but works for others. But, you could use --webroot. Just add a deploy-hook to reload Apache to pick up fresh cert like this:

certbot certonly --webroot -w /var/www/theforest -d forum.survivetheforest.net --deploy-hook apachectl graceful

Note no dry-run this time so should get fresh cert and update renewal conf

And, substitute your preferred reload/restart command for the one I used :slight_smile:

3 Likes
15

Produces this:

root@server:/var/www# certbot certonly --webroot -w /var/www/theforest -d forum.survivetheforest.net --deploy-hook apachectl graceful
usage:
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate.
certbot: error: unrecognized arguments: graceful
root@server:/var/www#

My certbot doesn't seem to have a --deploy-hook?

root@server:/var/www# certbot --help

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate. The most common SUBCOMMANDS and flags are:

obtain, install, and renew certificates:
    (default) run   Obtain & install a certificate in your current webserver
    certonly        Obtain or renew a certificate, but do not install it
    renew           Renew all previously obtained certificates that are near
expiry
    enhance         Add security enhancements to your existing configuration
   -d DOMAINS       Comma-separated list of domains to obtain a certificate for

  --apache          Use the Apache plugin for authentication & installation
  --standalone      Run a standalone webserver for authentication
  --nginx           Use the Nginx plugin for authentication & installation
  --webroot         Place files in a server's webroot folder for authentication
  --manual          Obtain certificates interactively, or using shell script
hooks

   -n               Run non-interactively
  --test-cert       Obtain a test certificate from a staging server
  --dry-run         Test "renew" or "certonly" without saving any certificates
to disk

manage certificates:
    certificates    Display information about certificates you have from Certbot
    revoke          Revoke a certificate (supply --cert-name or --cert-path)
    delete          Delete a certificate (supply --cert-name)
    reconfigure     Update a certificate's configuration (supply --cert-name)

manage your account:
    register        Create an ACME account
    unregister      Deactivate an ACME account
    update_account  Update an ACME account
    show_account    Display account details
  --agree-tos       Agree to the ACME server's Subscriber Agreement
   -m EMAIL         Email address for important account notifications

More detailed help:

  -h, --help [TOPIC]    print this message, or detailed help on a topic;
                        the available TOPICS are:

   all, automation, commands, paths, security, testing, or any of the
   subcommands or plugins (certonly, renew, install, register, nginx,
   apache, standalone, webroot, etc.)
  -h all                print a detailed help page including all topics
  --version             print the version number
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
root@server:/var/www#

However apache2 knows graceful

root@server:/var/www# apache2ctl --help
Usage: /usr/sbin/apache2ctl start|stop|restart|graceful|graceful-stop|configtest|status|fullstatus|help
       /usr/sbin/apache2ctl <apache2 args>
       /usr/sbin/apache2ctl -h            (for help on <apache2 args>)
root@server:/var/www#
1 Like
16

Sorry, probably need quotes around the deploy-hook command like 'apache2ctl graceful'

3 Likes
17

The document root folder of your web server. For Apache, it should match the DocumentRoot folder for that domain

3 Likes
18 
root@server:/var/www# certbot certonly --webroot -w /var/www/theforest -d forum.survivetheforest.net --deploy-hook "apachectl graceful"
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
An RSA certificate named forum.survivetheforest.net already exists. Do you want
to update its key type to ECDSA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(U)pdate key type/(K)eep existing key type: U
Renewing an existing certificate for forum.survivetheforest.net

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/forum.survivetheforest.net/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/forum.survivetheforest.net/privkey.pem
This certificate expires on 2023-06-07.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
root@server:/var/www#

Pheewwwww :sweat_smile: :sweat_smile: :sweat_smile:

Not seeing the certificate active after ctrl+f5 yet, same for forum.stranded-games but for the other 2 sites my browser was able to grab the renewed certs.

The renewal conf also looks different now - different than the other 3 :smiley:

root@server:/var/www# cat /etc/letsencrypt/renewal/forum.survivetheforest.net.conf
# renew_before_expiry = 30 days
version = 2.4.0
archive_dir = /etc/letsencrypt/archive/forum.survivetheforest.net
cert = /etc/letsencrypt/live/forum.survivetheforest.net/cert.pem
privkey = /etc/letsencrypt/live/forum.survivetheforest.net/privkey.pem
chain = /etc/letsencrypt/live/forum.survivetheforest.net/chain.pem
fullchain = /etc/letsencrypt/live/forum.survivetheforest.net/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = webroot
account = 95eea18f2a0f38811f9d069308e1b3a0
server = https://acme-v02.api.letsencrypt.org/directory
renew_hook = apachectl graceful
webroot_path = /var/www/theforest,
key_type = ecdsa
[[webroot_map]]
forum.survivetheforest.net = /var/www/theforest
root@server:/var/www#

What exactly does the -w flag in the command do?

1 Like
19

You need to fully restart Apache this time. I sometimes see fresh cert and sometimes old one (link here) . Sometimes an Apache worker gets stuck and needs restart. Or, sometimes even server reboot.

Maybe this stuck worker was why --apache wasn't working right for this domain. Just a guess.

The -w (--webroot-path) option is fully explained here
https://eff-certbot.readthedocs.io/en/stable/using.html#webroot

4 Likes
20

Oh wow, I had this already once that certs were renewed but I didn't see the renewed ones (before expiry browser's however got the new ones) and assumed this was a browser thing lol

This right here explained the -w entirely to me:

(...) by including certonly and --webroot on the command line. In addition, you’ll need to specify --webroot-path or -w with the top-level directory (“web root”) containing the files served by your webserver

Thank you a lot Mike for your help. Not only within almost no time, but also: what an insane quality support. Wow, totally impressed! Thank you so much for lending your hand :slight_smile: :cat:

3 Likes