(snap) Everything seems to install, but 443 doesn't work

Cheddar · September 29, 2020, 7:56pm

Okay, sounds easier to do it manually per your suggestion than to beat this horse for ever. I will give that a shot, and jump back on here. May not be able to do it until later on, or tomorrow though. I will report back! Thank you for your help so far; appreciated.

_az · September 29, 2020, 8:23pm

Is this not the problem?

How come your port 80 bind is on all addresses, but port 443 is on a network that has nothing to do with your server?

Change it from:

<VirtualHost 172.10.1.2:443>

to

<VirtualHost *:443>

rg305 · September 29, 2020, 8:32pm

I really don't see how that is the problem.
But I do agree that it should be either the correct IP or none at all [or it may never get used].
That said, do you even really need/use this file?
It might not hurt to try it once without that file altogether.

Cheddar · September 29, 2020, 8:36pm

Not entirely sure of exactly what you guys are indicating, but FYI, the Apache server is used as a local server (i.e. local 172* address) for internal stuff, and it also has a public IP for ukdirectsale.

Tried adding the Certificate Path bits, as given, but server errors with "Invalid command 'Certificate', perhaps misspelled or defined by a module not included in the server configuration". I guess those were not supposed to be copied verbatim.

_az · September 29, 2020, 8:38pm

172.10.1.2 is not a private address. It is assigned and publicly routable.

There's a test you can do to see if the bind address is the problem, or whether it's an external firewall:

openssl s_client -connect localhost:443 1>/dev/null

versus

openssl s_client -connect 172.10.1.2:443 1>/dev/null

do either of them work?

Cheddar · September 29, 2020, 8:41pm

$ sudo openssl s_client -connect localhost:443 1>/dev/null
depth=0 C = --, ST = SomeState, L = SomeCity, O = SomeOrganization, OU = SomeOrganizationalUnit, CN = dgbserver, emailAddress = root@dgbserver
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = --, ST = SomeState, L = SomeCity, O = SomeOrganization, OU = SomeOrganizationalUnit, CN = dgbserver, emailAddress = root@dgbserver
verify error:num=21:unable to verify the first certificate
verify return:1

read:errno=0


$ sudo openssl s_client -connect 172.10.1.2:443 1>/dev/null
depth=0 C = --, ST = SomeState, L = SomeCity, O = SomeOrganization, OU = SomeOrganizationalUnit, CN = dgbserver, emailAddress = root@dgbserver
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = --, ST = SomeState, L = SomeCity, O = SomeOrganization, OU = SomeOrganizationalUnit, CN = dgbserver, emailAddress = root@dgbserver
verify error:num=21:unable to verify the first certificate
verify return:1

read:errno=0

_az · September 29, 2020, 8:41pm

OK.

So Apache is working fine, but you have an external firewall blocking access to TCP 443, as @rg305 suggested earlier.

As to why Certbot couldn't deploy the certificate to Apache, I'd be curious to see what the full complaint is from:

certbot install --apache --cert-name ukdirectsale.co.uk

It's possible it just didn't work because your Apache configuration was busted initially, but after you fixed it, you might find the command has some success.

rg305 · September 29, 2020, 8:44pm

Those lines weren't to be used directly as entries in your code.
They merely showed the files that would need to be used.
If you are unfamiliar with TLS enabling a site, I can try to draft you a simple example or you could do a web search for plenty of context/tutorials.

Here is a very very basic example:

<VirtualHost *:443>
 DocumentRoot /your/doc/root/path
 ServerName ukdirectsale.co.uk
 ServerAlias www.ukdirectsale.co.uk
 Protocols h2 http/1.1
 SSLEngine On
 SSLOptions +StrictRequire
 SSLCertificateFile      /etc/letsencrypt/live/ukdirectsale.co.uk/fullchain.pem
 SSLCertificateKeyFile   /etc/letsencrypt/live/ukdirectsale.co.uk/privkey.pem
 SSLProtocol +TLSv1.2
 SSLCipherSuite HIGH:!3DES:!aNULL:!eNULL:!RC4
</VirtualHost>

Cheddar · September 29, 2020, 8:46pm

Alright .... But the external firewall was amended to allow 443 traffic. I'll take another look at it though, but I did check it over a couple of times already.

Even so, even if the firewall is somehow an issue, httpd.conf will still need tweaking anyway I guess. Do I need to add stuff like "SSLEngine on" to the conf file, or will certbot have already cleverly taken care of this somehow (even though none of the conf files appear to have changed)?

Edit Okay, posted after I read your most recent post.

_az · September 29, 2020, 8:47pm

The configuration wouldn't have changed because when Certbot tried to do it the first time, it failed ^^.

You can try manually configure it like @rg305 suggested, or using something like https://ssl-config.mozilla.org/.

You can also try get Certbot to do it again, using:

rg305 · September 29, 2020, 8:47pm

Certbot was unable to take care of anything.
It has only been able to get a cert (thus far).

Yeah this is the right test now:

Cheddar · September 29, 2020, 8:48pm

Okay.

$ sudo /var/lib/snapd/snap/bin/certbot install --apache --cert-name ukdirectsale.co.uk
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator None, Installer apache
Created an SSL vhost at /etc/httpd/conf.modules.d/dgb.vhosts-le-ssl.conf
Deploying Certificate to VirtualHost /etc/httpd/conf.modules.d/dgb.vhosts-le-ssl.conf
Created an SSL vhost at /etc/httpd/conf.modules.d/dgb.vhosts-le-ssl.conf
Deploying Certificate to VirtualHost /etc/httpd/conf.modules.d/dgb.vhosts-le-ssl.conf
An unexpected error occurred:
augeas.AugeasValueError: Augeas.set() failed: Too many matches for path expression
Please see the logfiles in /var/log/letsencrypt for more details.

IMPORTANT NOTES:
 - Unable to install the certificate

rg305 · September 29, 2020, 8:49pm

Sounds like we need to see that log file.

rg305 · September 29, 2020, 8:52pm

I think the problem is that certbot is unable to realize, and handle, multiple matching server blocks within the same file name.
The multiple outputs are all being directed to the same destination file and overwrite/overlap with each other.
So it's just not smart enough to create one destination file with multiple server blocks in it.

rg305 · September 29, 2020, 8:53pm

They are all three in the same file.

Try breaking them out into individual files.

_az · September 29, 2020, 8:56pm

It would indeed be good if you could post the log file, because then we could file a bug report. Interestingly it's not a issue that has been reported before.

Otherwise I'd probably continue by configuring Apache using https://ssl-config.mozilla.org/ or whatever, until we know for sure what causes the issue.

Cheddar · September 29, 2020, 8:56pm

Yes, okay, I can try splitting into multiple files, but that'll have to wait till tomorrow.

The log file looks like .... this:

# cat letsencrypt.log
2020-09-29 16:49:18,673:DEBUG:certbot._internal.main:certbot version: 1.8.0
2020-09-29 16:49:18,673:DEBUG:certbot._internal.main:Arguments: ['--apache', '--cert-name', 'ukdirectsale.co.uk']
2020-09-29 16:49:18,674:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2020-09-29 16:49:18,701:DEBUG:certbot._internal.log:Root logging level set at 20
2020-09-29 16:49:18,702:INFO:certbot._internal.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2020-09-29 16:49:18,703:DEBUG:certbot._internal.plugins.selection:Requested authenticator apache and installer apache
2020-09-29 16:49:18,863:DEBUG:certbot_apache._internal.configurator:Apache version is 2.4.6
2020-09-29 16:49:19,305:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * apache
Description: Apache Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache._internal.entrypoint:ENTRYPOINT
Initialized: <certbot_apache._internal.override_centos.CentOSConfigurator object at 0x7fcec31c6c40>
Prep: True
2020-09-29 16:49:19,306:DEBUG:certbot._internal.plugins.selection:Selected authenticator None and installer <certbot_apache._internal.override_centos.CentOSConfigurator object at 0x7fcec31c6c40>
2020-09-29 16:49:19,306:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator None, Installer apache
2020-09-29 16:49:19,334:INFO:certbot_apache._internal.configurator:Created an SSL vhost at /etc/httpd/conf.modules.d/dgb.vhosts-le-ssl.conf
2020-09-29 16:49:19,392:DEBUG:certbot.reverter:Creating backup of /etc/httpd/conf.modules.d/dgb.vhosts-le-ssl.conf
2020-09-29 16:49:19,585:INFO:certbot_apache._internal.configurator:Deploying Certificate to VirtualHost /etc/httpd/conf.modules.d/dgb.vhosts-le-ssl.conf
2020-09-29 16:49:19,697:INFO:certbot_apache._internal.configurator:Created an SSL vhost at /etc/httpd/conf.modules.d/dgb.vhosts-le-ssl.conf
2020-09-29 16:49:19,959:INFO:certbot_apache._internal.configurator:Deploying Certificate to VirtualHost /etc/httpd/conf.modules.d/dgb.vhosts-le-ssl.conf
2020-09-29 16:49:20,071:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot/_internal/client.py", line 516, in deploy_certificate
    self.installer.deploy_cert(
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot_apache/_internal/configurator.py", line 500, in deploy_cert
    vhosts = self.choose_vhosts(domain)
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot_apache/_internal/configurator.py", line 524, in choose_vhosts
    return [self.choose_vhost(domain, create_if_no_ssl)]
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot_apache/_internal/configurator.py", line 695, in choose_vhost
    self._add_servername_alias(target_name, vhost)
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot_apache/_internal/configurator.py", line 1686, in _add_servername_alias
    self.parser.add_dir(vh_path, "ServerAlias", target_name)
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot_apache/_internal/parser.py", line 441, in add_dir
    self.aug.set(aug_conf_path + "/directive[last() + 1]", directive)
  File "/snap/certbot/579/lib/python3.8/site-packages/augeas/__init__.py", line 263, in set
    self._raise_error(AugeasValueError, "Augeas.set() failed")
  File "/snap/certbot/579/lib/python3.8/site-packages/augeas/__init__.py", line 154, in _raise_error
    raise errorclass(ec, fullmessage, msg, minor, details)
augeas.AugeasValueError: Augeas.set() failed: Too many matches for path expression

2020-09-29 16:49:20,072:DEBUG:certbot._internal.error_handler:Calling registered functions
2020-09-29 16:49:20,073:DEBUG:certbot._internal.reporter:Reporting to user: Unable to install the certificate
2020-09-29 16:49:20,074:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/snap/certbot/579/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot/main.py", line 15, in main
    return internal_main.main(cli_args)
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot/_internal/main.py", line 1358, in main
    return config.func(config, plugins)
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot/_internal/main.py", line 808, in install
    _install_cert(config, le_client, domains)
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot/_internal/main.py", line 757, in _install_cert
    le_client.deploy_certificate(domains, path_provider.key_path,
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot/_internal/client.py", line 516, in deploy_certificate
    self.installer.deploy_cert(
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot_apache/_internal/configurator.py", line 500, in deploy_cert
    vhosts = self.choose_vhosts(domain)
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot_apache/_internal/configurator.py", line 524, in choose_vhosts
    return [self.choose_vhost(domain, create_if_no_ssl)]
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot_apache/_internal/configurator.py", line 695, in choose_vhost
    self._add_servername_alias(target_name, vhost)
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot_apache/_internal/configurator.py", line 1686, in _add_servername_alias
    self.parser.add_dir(vh_path, "ServerAlias", target_name)
  File "/snap/certbot/579/lib/python3.8/site-packages/certbot_apache/_internal/parser.py", line 441, in add_dir
    self.aug.set(aug_conf_path + "/directive[last() + 1]", directive)
  File "/snap/certbot/579/lib/python3.8/site-packages/augeas/__init__.py", line 263, in set
    self._raise_error(AugeasValueError, "Augeas.set() failed")
  File "/snap/certbot/579/lib/python3.8/site-packages/augeas/__init__.py", line 154, in _raise_error
    raise errorclass(ec, fullmessage, msg, minor, details)
augeas.AugeasValueError: Augeas.set() failed: Too many matches for path expression
2020-09-29 16:49:20,074:ERROR:certbot._internal.log:An unexpected error occurred:
2020-09-29 16:49:20,075:ERROR:certbot._internal.log:augeas.AugeasValueError: Augeas.set() failed: Too many matches for path expression

Cheddar · September 30, 2020, 3:21pm

Okay, making some headway. I'm still playing around a bit, as it didn't quite work properly when I did a permanent redirect on each vhost (the main one, and the images* one).

Still, the several things I did do were to remove duplicate module entries (good housekeeping though I don't think it's related). Manually added the ssl blocks to the new vhost entries (i.e. keeping separate vhosts for 80 and 443), and then played around with the firewall. Yesterday I had amended the router firewall to allow 443, but omitted allowing CentOS to do the same, so that's now fixed.

Anyway, I'll play around some more, as still a few bits and pieces to do with mixed secure/insecure content.

Thanks for the assistance to date!

system · October 30, 2020, 3:21pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.