Apache plugin is installed, is not installed

edwinm · October 6, 2020, 7:43pm

I ran this command:
sudo certbot certonly --apache --cert-name bitstorm -d bitstorm.org -d www.bitstorm.org

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for bitstorm.org
http-01 challenge for www.bitstorm.org
Waiting for verification...
Challenge failed for domain bitstorm.org
Challenge failed for domain www.bitstorm.org
http-01 challenge for bitstorm.org
http-01 challenge for www.bitstorm.org
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:

The following errors were reported by the server:

Domain: bitstorm.org
Type: unauthorized
Detail: Invalid response from
http://bitstorm.org/.well-known/acme-challenge/w15zovBEbUwhBtk5BtKl3-L7NpNcSybMRpmGpOGf-dw
[83.162.203.56]: "\n\n404 Not
Found\n\n

Not Found
\n<p"
Domain: www.bitstorm.org
Type: unauthorized
Detail: Invalid response from
http://www.bitstorm.org/.well-known/acme-challenge/kIpbBITN8YU08byw8A_why-GzCDvzhEC0Z1OVafkMA0
[83.162.203.56]: "\n\n404 Not
Found\n\n

Not Found
\n<p"
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

My web server is (include version):
Server version: Apache/2.4.41 (Ubuntu)

The operating system my web server runs on is (include version):
Ubuntu 20.04.1 LTS

I can login to a root shell on my machine (yes or no, or I don't know):
yes

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 0.40.0

Installed apache plugin: python3-certbot-apache

Strange thing is, sudo certbot plugins returns apache plugin, but logfile says:

2020-10-04 11:42:56,728:INFO:certbot.renewal:Cert is due for renewal, auto-renewing...
2020-10-04 11:42:56,728:INFO:certbot.renewal:Non-interactive renewal: random delay of 459.2428117522985 seconds
2020-10-04 11:50:36,068:DEBUG:certbot.plugins.selection:Requested authenticator apache and installer apache
2020-10-04 11:50:36,068:DEBUG:certbot.plugins.selection:No candidate plugin
2020-10-04 11:50:36,068:DEBUG:certbot.plugins.selection:No candidate plugin
2020-10-04 11:50:36,068:DEBUG:certbot.plugins.selection:Selected authenticator None and installer None
2020-10-04 11:50:36,068:INFO:certbot.main:Could not choose appropriate plugin: The requested apache plugin does not appear to be installed
2020-10-04 11:50:36,069:WARNING:certbot.renewal:Attempting to renew cert (bitstorm) from /etc/letsencrypt/renewal/bitstorm.conf produced an unexpected error: The requested apache plugin does not appear to be installed. Skipping.

JuergenAuer · October 6, 2020, 7:54pm

Hi @edwinm

if you have that error, Certbot doesn't understand your config.

What says

apachectl -S

Are there duplicated combinations port 80 + domain names?

Osiris · October 6, 2020, 8:06pm

It looks like this is a second issue, besides the first issue @JuergenAuer already addressed. The latter part of your post looks like it was started from a cronjob? Because it says "Non-interactive renewal". Could it be that certbot runs in a different environment?

rg305 · October 6, 2020, 8:19pm

Please show this file:

EDIT: Without knowing anything else, I would say you might benefit from replacing your current certbot with the latest version from snaps.
[replace here means to properly apt remove the existing certbot and components first]

edwinm · October 6, 2020, 8:26pm

Osiris: Both commands are run from the same command line.

Output of apachectl -S:

VirtualHost configuration:
192.168.0.23:443       is a NameVirtualHost
         port 443 namevhost bitstorm.org (/etc/apache2/sites-enabled/www.bitstorm.org-le-ssl.conf:4)
                 alias www.bitstorm.org
*:80                   bitstorm.org (/etc/apache2/sites-enabled/www.bitstorm.org.conf:1)
AH00515: WARNING: MaxRequestWorkers of 500 would require 20 servers and would exceed ServerLimit of 16, decreasing to 400. To increase, please see the ServerLimit directive.
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex rewrite-map: using_defaults
Mutex ssl-stapling-refresh: using_defaults
Mutex fcgid-proctbl: using_defaults
Mutex ssl-stapling: using_defaults
Mutex proxy: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/run/apache2/" mechanism=default
Mutex fcgid-pipe: using_defaults
Mutex watchdog-callback: using_defaults
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="www-data" id=33 not_used
Group: name="www-data" id=33 not_used

cat /etc/letsencrypt/renewal/bitstorm.conf

# renew_before_expiry = 30 days
cert = /etc/letsencrypt/live/bitstorm/cert.pem
privkey = /etc/letsencrypt/live/bitstorm/privkey.pem
chain = /etc/letsencrypt/live/bitstorm/chain.pem
fullchain = /etc/letsencrypt/live/bitstorm/fullchain.pem
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/bitstorm

# Options and defaults used in the renewal process
[renewalparams]
authenticator = apache
account = 5d24ff87814951619b6cb820371a11d5
server = https://acme-v02.api.letsencrypt.org/directory
installer = apache

Osiris · October 6, 2020, 8:29pm

What was the exact command for the latter part of your post then?

JuergenAuer · October 6, 2020, 8:29pm

There

edwinm:

VirtualHost configuration:
192.168.0.23:443       is a NameVirtualHost
         port 443 namevhost bitstorm.org (/etc/apache2/sites-enabled/www.bitstorm.org-le-ssl.conf:4)
                 alias www.bitstorm.org
*:80                   bitstorm.org (/etc/apache2/sites-enabled/www.bitstorm.org.conf:1)

you see the problem.

The port 80 vHost doesn't have the www domain name. So there is no matching vHost of your command.

edwinm · October 6, 2020, 8:45pm

sudo certbot plugins

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* apache
Description: Apache Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT

* standalone
Description: Spin up a temporary webserver
Interfaces: IAuthenticator, IPlugin
Entry point: standalone = certbot.plugins.standalone:Authenticator

* webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

The output of /var/log/letsencrypt/letsencrypt.log is the same as mentioned earlier. So the same command that prints the existence of the apache plugin writes a log about missing the apache plugin.

Osiris · October 6, 2020, 8:49pm

I'm doubting that very much. Are you checking the correct log file? Because the log file in your first post showed a renewal attempt. That would never be in a certbot plugins command.

edwinm · October 6, 2020, 8:55pm

This is plain copy and paste of my terminal:

[edwin:~]$ sudo certbot plugins
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* apache
Description: Apache Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT

* standalone
Description: Spin up a temporary webserver
Interfaces: IAuthenticator, IPlugin
Entry point: standalone = certbot.plugins.standalone:Authenticator

* webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[edwin:~]$ sudo ls -l /var/log/letsencrypt/letsencrypt.log
-rw-r--r-- 1 root root 576517 Oct  6 20:54 /var/log/letsencrypt/letsencrypt.log
[edwin:~]$ date
Tue 06 Oct 2020 08:54:17 PM UTC
[edwin:~]$ sudo head /var/log/letsencrypt/letsencrypt.log
2020-10-04 11:42:56,705:DEBUG:certbot.main:certbot version: 0.40.0
2020-10-04 11:42:56,705:DEBUG:certbot.main:Arguments: ['-q']
2020-10-04 11:42:56,705:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2020-10-04 11:42:56,713:DEBUG:certbot.log:Root logging level set at 30
2020-10-04 11:42:56,713:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2020-10-04 11:42:56,720:DEBUG:certbot.plugins.selection:Requested authenticator <certbot.cli._Default object at 0x7f4f5ddf66d0> and installer <certbot.cli._Default object at 0x7f4f5ddf66d0>
2020-10-04 11:42:56,728:DEBUG:certbot.storage:Should renew, less than 30 days before certificate expiry 2020-10-19 15:22:24 UTC.
2020-10-04 11:42:56,728:INFO:certbot.renewal:Cert is due for renewal, auto-renewing...
2020-10-04 11:42:56,728:INFO:certbot.renewal:Non-interactive renewal: random delay of 459.2428117522985 seconds
2020-10-04 11:50:36,068:DEBUG:certbot.plugins.selection:Requested authenticator apache and installer apache

Osiris · October 6, 2020, 8:56pm

See the date? That's 2 days ago.

edwinm · October 6, 2020, 9:00pm

Oh, thanks, I looked over this! I'm not used to debug info to append to the logfile.

It does find the apache plugin.

2020-10-06 20:54:07,969:DEBUG:certbot.main:certbot version: 0.40.0
2020-10-06 20:54:07,970:DEBUG:certbot.main:Arguments: []
2020-10-06 20:54:07,970:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2020-10-06 20:54:07,983:DEBUG:certbot.log:Root logging level set at 20
2020-10-06 20:54:07,983:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2020-10-06 20:54:07,984:DEBUG:certbot.main:Expected interfaces: None
2020-10-06 20:54:07,984:DEBUG:certbot.main:Filtered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#standalone,PluginEntryPoint#webroot)

edwinm · October 6, 2020, 9:02pm

@JuergenAuer I do have an ServerAlias www.bitstorm.org. I will investigate why it's not showing with apachectl -S.

rg305 · October 7, 2020, 12:51am

Please show this file:

And have you considered replacing your current certbot with the latest version from snaps ?

edwinm · October 16, 2020, 8:56pm

Here's an update.

I installed the snap version of certbot, but that didn't solve the issue.

Then I went over the rate limit and couldn't investigage further.

Then, 30 minutes later, I ran the command again and it suddenly worked!

I'm glad it is working again, but my confidence in the software has dropped a bit.

Thanks for your help.

rg305 · October 16, 2020, 8:59pm

You should really understand that part of the problem is more than likely in the Apache config.

[&2* readers: Get involved; Be heard. It starts with: if you read something you like, then like it ]

Apache plugin is installed, is not installed

Not Found

Not Found