Certbot: Unable to install the certificate into Apache after deleting the VirthualHost configuration files

It isn’t very talkative:

# Because we are using logrotate for greater flexibility, disable the
# internal certbot logrotation.
2 Likes

Speechless, just the way I like that file.

Hmm...

3 Likes

I really hope I'm wrong about this...

What contains /var/log/letsencrypt?

3 Likes
ll /var/log/letsencrypt/
insgesamt 236
-rw-r--r-- 1 root root 236850 Okt  3 19:58 letsencrypt.log
2 Likes

Just for completeness...

What contains /etc/letsencrypt/options-ssl-apache.conf?

3 Likes
# This file contains important security parameters. If you modify this file
# manually, Certbot will be unable to automatically provide future security
# updates. Instead, Certbot will print and log an error message with a path to
# the up-to-date file that you will need to refer to when manually updating
# this file.

SSLEngine on

# Intermediate configuration, tweak to your needs
SSLProtocol             all -SSLv2 -SSLv3
SSLCipherSuite          ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS
SSLHonorCipherOrder     on
SSLCompression          off

SSLOptions +StrictRequire

# Add vhost name to log entries:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" vhost_combined
LogFormat "%v %h %l %u %t \"%r\" %>s %b" vhost_common

#CustomLog /var/log/apache2/access.log vhost_combined
#LogLevel warn
#ErrorLog /var/log/apache2/error.log

# Always ensure Cookies have "Secure" set (JAH 2012/1)
#Header edit Set-Cookie (?i)^(.*)(;\s*secure)??((\s*;)?(.*)) "$1; Secure$3$4"
2 Likes

Looking good there.

Please rename letsencrypt.log to old.log then run:

certbot run --cert-name photo --apache --keep

3 Likes
# certbot run --cert-name photo --apache --keep
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Cert not yet due for renewal
Keeping the existing certificate
File: 
 - Could not be found to be deleted /etc/apache2/sites-available/martinwurm.photo-le-ssl.conf - Certbot probably shut down unexpectedly
An unexpected error occurred:
StopIteration
Please see the logfiles in /var/log/letsencrypt for more details.

IMPORTANT NOTES:
 - Unable to install the certificate
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/photo/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/photo/privkey.pem
   Your cert will expire on 2021-01-01. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot again
   with the "certonly" option. To non-interactively renew *all* of
   your certificates, run "certbot renew"

The content of the new letsencrypt.log:

2020-10-03 20:30:42,830:DEBUG:certbot.main:certbot version: 0.31.0
2020-10-03 20:30:42,830:DEBUG:certbot.main:Arguments: ['--cert-name', 'photo', '--apache', '--keep']
2020-10-03 20:30:42,831:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2020-10-03 20:30:42,841:DEBUG:certbot.log:Root logging level set at 20
2020-10-03 20:30:42,842:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2020-10-03 20:30:42,843:DEBUG:certbot.plugins.selection:Requested authenticator apache and installer apache
2020-10-03 20:30:43,004:DEBUG:certbot_apache.configurator:Apache version is 2.4.38
2020-10-03 20:30:43,343:DEBUG:certbot.plugins.selection:Single candidate plugin: * apache
Description: Apache Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT
Initialized: <certbot_apache.override_debian.DebianConfigurator object at 0x7f9e94b7aef0>
Prep: True
2020-10-03 20:30:43,350:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_apache.override_debian.DebianConfigurator object at 0x7f9e94b7aef0> and installer <certbot_apache.override_debian.DebianConfigurator object at 0x7f9e94b7aef0>
2020-10-03 20:30:43,354:INFO:certbot.plugins.selection:Plugins selected: Authenticator apache, Installer apache
2020-10-03 20:30:43,358:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-v02.api.letsencrypt.org/acme/acct/98301579', new_authzr_uri=None, terms_of_service=None), [do you need that? it looked like it’s not supposed to be on the internet], Meta(creation_dt=datetime.datetime(2020, 10, 3, 14, 34, 24, tzinfo=<UTC>), creation_host='martinwurm.photography'))>
2020-10-03 20:30:43,360:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2020-10-03 20:30:43,362:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org:443
2020-10-03 20:30:44,939:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 658
2020-10-03 20:30:44,941:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Sat, 03 Oct 2020 18:30:44 GMT
Content-Type: application/json
Content-Length: 658
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "0pbVyy9ENhU": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org"
  },
  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}
2020-10-03 20:30:44,951:INFO:certbot.renewal:Cert not yet due for renewal
2020-10-03 20:30:44,952:INFO:certbot.main:Keeping the existing certificate
2020-10-03 20:30:44,953:DEBUG:certbot.reporter:Reporting to user: Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/photo/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/photo/privkey.pem
Your cert will expire on 2021-01-01. To obtain a new or tweaked version of this certificate in the future, simply run certbot again with the "certonly" option. To non-interactively renew *all* of your certificates, run "certbot renew"
2020-10-03 20:30:44,967:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 516, in deploy_certificate
    fullchain_path=fullchain_path)
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 334, in deploy_cert
    vhosts = self.choose_vhosts(domain)
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 358, in choose_vhosts
    return [self.choose_vhost(domain, create_if_no_ssl)]
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 540, in choose_vhost
    vhost = self.make_vhost_ssl(vhost)
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 1116, in make_vhost_ssl
    self._copy_create_ssl_vhost_skeleton(nonssl_vhost, ssl_fp)
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 1260, in _copy_create_ssl_vhost_skeleton
    ssl_vh_contents, sift = self._sift_rewrite_rules(orig_contents)
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 1332, in _sift_rewrite_rules
    line = next(contents)
StopIteration

2020-10-03 20:30:44,967:DEBUG:certbot.error_handler:Calling registered functions
2020-10-03 20:30:44,968:WARNING:certbot.reverter:File: 
 - Could not be found to be deleted /etc/apache2/sites-available/martinwurm.photo-le-ssl.conf - Certbot probably shut down unexpectedly
2020-10-03 20:30:44,970:DEBUG:certbot.reporter:Reporting to user: Unable to install the certificate
2020-10-03 20:30:44,970:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.31.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1365, in main
    return config.func(config, plugins)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1126, in run
    _install_cert(config, le_client, domains, new_lineage)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 760, in _install_cert
    path_provider.cert_path, path_provider.chain_path, path_provider.fullchain_path)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 516, in deploy_certificate
    fullchain_path=fullchain_path)
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 334, in deploy_cert
    vhosts = self.choose_vhosts(domain)
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 358, in choose_vhosts
    return [self.choose_vhost(domain, create_if_no_ssl)]
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 540, in choose_vhost
    vhost = self.make_vhost_ssl(vhost)
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 1116, in make_vhost_ssl
    self._copy_create_ssl_vhost_skeleton(nonssl_vhost, ssl_fp)
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 1260, in _copy_create_ssl_vhost_skeleton
    ssl_vh_contents, sift = self._sift_rewrite_rules(orig_contents)
  File "/usr/lib/python3/dist-packages/certbot_apache/configurator.py", line 1332, in _sift_rewrite_rules
    line = next(contents)
StopIteration
2020-10-03 20:30:44,972:ERROR:certbot.log:An unexpected error occurred:
2 Likes

You need not worry about protecting your ACME account number. The account private key is the secret, which would never be in the log file.

3 Likes

I figured, but I thought you can never be too careful if you don’t know for certain. :slightly_smiling_face: Do you need that number?

1 Like

The account number is in the directory name. :slightly_smiling_face:

https://acme-v02.api.letsencrypt.org/acme/acct/98301579

3 Likes

That wasn’t what I removed, though. I removed a ~30 digit hex number.

1 Like

Ah... probably just a signature. Not great to show, but it contains a nonce, so not like it will be useful to an attacker.

3 Likes

Let's take a look in /etc/letsencrypt/accounts/

We need to exercise caution here.

2 Likes
tree
.
└── acme-v02.api.letsencrypt.org
    └── directory
        └── [the same hex number I removed ealier]
            ├── meta.json
            ├── private_key.json
            └── regr.json

3 directories, 3 files
1 Like

Could be the thumbprint. :thinking:

That shouldn't be in the log if that's the case.

No need to dig further there anyway.

2 Likes

I’m impressed at how effectively I seem to have mangled this thing up by deleting just that one file on a fresh install. :smile:

I’m starting to consider just nuking the whole thing and starting over, but it might be helpful to keep it for now just in case the source of the error can aid the devs. What do you think?

2 Likes

You read my mind. :exploding_head:

You really have nothing to lose. You can try uninstalling and reinstalling certbot first just for the sake of trial. You can only generate one more new certificate though until you get rate-limited. If that happens, come back here and I'll get you another certificate.

As for the devs, you better believe I want them to look this over. :grin:

@certbot-devs

I think you guys have about everything under the sun available to you in this topic. A little help here, please. Would be really nice to know the cause of this and how to fix it in future.

2 Likes

All right. It’s not a time-critical project, so I’ll just leave the server as it is in case you want to keep digging. :smile:

2 Likes

Thanks for your patience. We're all still learning. :upside_down_face:

3 Likes