Certbot renew fails under RHEL 9

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:
multiple

I ran this command:
certbot renewcertbot

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/2pi.net.conf


Error while running apachectl configtest.

Usage: apachectl [-D name] [-d directory] [-f file]
[-C "directive"] [-c "directive"]
[-k start|restart|graceful|graceful-stop|stop]
[-v] [-V] [-h] [-l] [-L] [-t] [-T] [-S] [-X]
Options:
-D name : define a name for use in directives
-d directory : specify an alternate initial ServerRoot
-f file : specify an alternate ServerConfigFile
-C "directive" : process directive before reading config files
-c "directive" : process directive after reading config files
-e level : show startup errors of level (see LogLevel)
-E file : log startup errors to file
-v : show version number
-V : show compile settings
-h : list available command line options (this page)
-l : list compiled in modules
-L : list available configuration directives
-t -D DUMP_VHOSTS : show parsed vhost settings
-t -D DUMP_RUN_CFG : show parsed run settings
-S : a synonym for -t -D DUMP_VHOSTS -D DUMP_RUN_CFG
-t -D DUMP_MODULES : show all loaded modules
-M : a synonym for -t -D DUMP_MODULES
-t -D DUMP_INCLUDES: show all included configuration files
-t : run syntax check for config files
-T : start without DocumentRoot(s) check
-X : debug mode (only one worker, do not detach)

Failed to renew certificate 2pi.net with error: The apache plugin is not working; there may be problems with your existing configuration.
The error was: MisconfigurationError('Error while running apachectl configtest.\n\nUsage: apachectl [-D name] [-d directory] [-f file]\n [-C "directive"] [-c "directive"]\n [-k start|restart|graceful|graceful-stop|stop]\n [-v] [-V] [-h] [-l] [-L] [-t] [-T] [-S] [-X]\nOptions:\n -D name : define a name for use in directives\n -d directory : specify an alternate initial ServerRoot\n -f file : specify an alternate ServerConfigFile\n -C "directive" : process directive before reading config files\n -c "directive" : process directive after reading config files\n -e level : show startup errors of level (see LogLevel)\n -E file : log startup errors to file\n -v : show version number\n -V : show compile settings\n -h : list available command line options (this page)\n -l : list compiled in modules\n -L : list available configuration directives\n -t -D DUMP_VHOSTS : show parsed vhost settings\n -t -D DUMP_RUN_CFG : show parsed run settings\n -S : a synonym for -t -D DUMP_VHOSTS -D DUMP_RUN_CFG\n -t -D DUMP_MODULES : show all loaded modules \n -M : a synonym for -t -D DUMP_MODULES\n -t -D DUMP_INCLUDES: show all included configuration files\n -t : run syntax check for config files\n -T : start without DocumentRoot(s) check\n -X : debug mode (only one worker, do not detach)\n')


All renewals failed. The following certificates could not be renewed:
/etc/letsencrypt/live/2pi.net/fullchain.pem (failure)


1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version):httpd-2.4.51-7.el9_0.x86_64

The operating system my web server runs on is (include version):
RHEL 9.0

My hosting provider, if applicable, is:
N/A

I can login to a root shell on my machine (yes or no, or I don't know):
Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
1.29.0

NOTE: I previously posted my comments under another posting, because the problem seems identical.

  1. Current versions of httpd/apache do not support fancy options with apachectl or apache2ctl. apachectl can only be used to start and stop apache. All other functions are deprecated.

  2. I make certbot use "httpd" instead of "apachectl" by two different methods:
    a) Replace apachectl with a link to httpd.
    b) certbot --apache-ctl httpd renew
    Both product the same result (above).

The output of httpd -t -D DUMP_RUN_CFG is

[Mon Sep 05 07:10:52.969150 2022] [so:warn] [pid 20292:tid 20292] AH01574: module headers_module is already loaded, skipping
ServerRoot: "/etc/httpd"
Main DocumentRoot: "/home/fulab/www/html"
Main ErrorLog: "/etc/httpd/logs/MYDOMAIN.com-error_log"
Mutex proxy-balancer-shm: using_defaults
Mutex ssl-stapling-refresh: using_defaults
Mutex authdigest-client: using_defaults
Mutex fcgid-proctbl: using_defaults
Mutex ssl-stapling: using_defaults
Mutex proxy: using_defaults
Mutex authn-socache: using_defaults
Mutex default: dir="/etc/httpd/run/" mechanism=default
Mutex mpm-accept: using_defaults
Mutex authdigest-opaque: using_defaults
Mutex watchdog-callback: using_defaults
Mutex rewrite-map: using_defaults
Mutex lua-ivm-shm: using_defaults
Mutex ssl-cache: using_defaults
Mutex cache-socache: using_defaults
Mutex fcgid-pipe: using_defaults
PidFile: "/etc/httpd/run/httpd.pid"
Define: DUMP_RUN_CFG
User: name="apache" id=48
Group: name="apache" id=48

The output of

httpd -t -D DUMP_MODULE is

[Mon Sep 05 07:12:42.666475 2022] [so:warn] [pid 20837:tid 20837] AH01574: module headers_module is already loaded, skipping
Syntax OK

Finally:

httpd -t -D DUMP_INCLUDES

produces the following output:

Included configuration files:
(*) /etc/httpd/conf/httpd.conf
(58) /etc/httpd/conf.modules.d/00-base.conf
(58) /etc/httpd/conf.modules.d/00-dav.conf
(58) /etc/httpd/conf.modules.d/00-lua.conf
(58) /etc/httpd/conf.modules.d/00-mpm.conf
(58) /etc/httpd/conf.modules.d/00-optional.conf
(58) /etc/httpd/conf.modules.d/00-proxy.conf
(58) /etc/httpd/conf.modules.d/00-ssl.conf
(58) /etc/httpd/conf.modules.d/00-systemd.conf
(58) /etc/httpd/conf.modules.d/01-cgi.conf
(58) /etc/httpd/conf.modules.d/10-fcgid.conf
(58) /etc/httpd/conf.modules.d/10-h2.conf
(58) /etc/httpd/conf.modules.d/10-proxy_h2.conf
(58) /etc/httpd/conf.modules.d/apreq.conf
[Mon Sep 05 07:57:44.549877 2022] [so:warn] [pid 33773:tid 33773] AH01574: module headers_module is already loaded, skipping
(189) /etc/httpd/conf.d/autoindex.conf
(189) /etc/httpd/conf.d/fcgid.conf
(189) /etc/httpd/conf.d/manual.conf
(189) /etc/httpd/conf.d/php.conf
(189) /etc/httpd/conf.d/ssl.conf
(189) /etc/httpd/conf.d/userdir.conf
(189) /etc/httpd/conf.d/welcome.conf
(1267) /etc/httpd/conf/httpd-le-ssl.conf
(20) /etc/letsencrypt/options-ssl-apache.conf
(43) /etc/letsencrypt/options-ssl-apache.conf
(92) /etc/letsencrypt/options-ssl-apache.conf
(174) /etc/letsencrypt/options-ssl-apache.conf
(207) /etc/letsencrypt/options-ssl-apache.conf
(228) /etc/letsencrypt/options-ssl-apache.conf
(252) /etc/letsencrypt/options-ssl-apache.conf
(270) /etc/letsencrypt/options-ssl-apache.conf

I have been using certbot for several years. It worked fine under Centos 8 and Centos 8 Stream. It does not work under RHEL 9.

One other factum: Getting certbot installed in RHEL 9 was non-trivial because it is not in the RHEL 9 EPEL libraries. I ended up using snap.

@certbot-devs is it possible that a new OS override entry is needed for this OS?

@jschatzman how did you install Apache on this system? Was it a default, supported method for your OS?

1 Like

Yes, tracked by --apache doesn't work on RHEL9, need to use `httpd` rather than `apachectl` · Issue #9386 · certbot/certbot · GitHub. However, since we have a release tomorrow and today is a public holiday in the US, the fix probably won't be available until the October release.

3 Likes

Thanks, @_az! Could you suggest a temporary workaround?

1 Like

I don't think a workaround is possible. I suggested in the earlier thread to use the --webroot plugin in the meantime.

1 Like

I'm puzzled why --apache-ctl httpd woult still lead Certbot to complain about using apachectl..?

2 Likes

Unfortunately, that causes apachectl configtest to become httpd configtest, which is not valid.

apachectl configtest is still valid but the commands to dump defines, includes and modules need to use httpd. There is currently no flag combination that can be passed to Certbot to achieve this.

3 Likes

Standard install with dnf/yum from the RHEL 9 repo.

It sounds like there is no simple fix until the October release. If there is a prerelease that becomes available for testing, I would be happy to try it out.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.