Have to manually create .well-known/acme-challenge?

I simply followed this https://certbot.eff.org/lets-encrypt/ubuntufocal-apache to create and install a cert on apache. But the “sudo certbot renew --dry-run” failed with error below.

Isn’t the certbot CLI supposed to create the .well-known/acme-challenge dir by parsing apache configs? Nowhere says I need to create it manually. I did anyway and can see it at

But the “sudo certbot renew --webroot-path /var/www/html/ --dry-run” still fails with the same error. Then I added the authenticator flag
“sudo certbot renew --webroot-path /var/www/html/ -a webroot --dry-run”. It finally succeeded.

So, couldn’t all these be automated? At least should be asked in questionnaires (webroot, authenticator) during the first time cert install so that the later renewal can be more smoothly done.

1 Like

Yes, if you know what you need.

It looks like there is some misalignment between your apache config and what certbot remebers. You can try with:

certbot renew --apache --dry-run

Only if you use the apache plugin...

1 Like

I did try “sudo certbot renew --dry-run --apache” and got the same error though.

1 Like

If this one succeeds, you can make the change permanent by running without --dry-run.

Then, in future, those same authenticator flags will be used automatically. This happens by updating the configuration file in /etc/letsencrypt/renewal/. (You can also update the file by hand, but it's not recommended).

Is that what you are asking about?

1 Like

I meant the correct renew CLI flags were after lots of trials and online search, because the initial cert creation had no indication of the need for these tweaks. It’d be saving all these confusions if the initial “sudo certbot --apache” asked about these flags.

1 Like

Well, the idea in certbot --apache is that it shouldn't require input from the user. It's supposed to automagically figure out your configuration and solve it. However, that clearly didn't work out for you. That is most likely a defect in Certbot, but the devil is always in the details.

When you used this command ...:

... you switched from --apache to --webroot.

The webroot authenticator, unlike the Apache authenticator, does require user input. Every environment is different, therefore the instructions can't tell you what webroot path to use.

A confusing experience for you either way, but I think primarily due to the fact that --apache didn't work for you. Maybe the docs can be more clear that there are two separate approaches you can try.

If you have a more concrete idea about what the instructions should say (in light of the above), it would be valued feedback.

1 Like

But not for the next renewal, in principal.

In any case, the reason why the --apache plugin doesn't work properly is mostly due to misconfigured Apache configurations.

1 Like

If you have an uncommon configuration, like DocumentRoot does not point to /var/www/html/ but some RewriteRule pushes unknown things back to it, it might be worth submitting your actual conf files as a bug. The ways you can use and abuse Apache configuration is almost turing-complete, but if it’s a real bug or feasible to support it could be fixed.

1 Like

Besides the standard apache ubuntu package, I installed nextcloud, with a config like this (my original was only the first section and those rewrites were added by certbot I believe):

Alias /nextcloud “/var/www/nextcloud/”

<Directory /var/www/nextcloud/>
Require all granted
AllowOverride All
Options FollowSymLinks MultiViews
Satisfy Any

Dav off

<VirtualHost *:80>
ServerName mydomain.com
Redirect permanent / https://mydomain.com/
RewriteEngine on
RewriteCond %{SERVER_NAME} =mydomain.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

Then I believe certbot created the default-ssl.conf as below. I had to commented out the self-signed Ubuntu certs and replace them with the new letsencrypt certs to make both the root site and nextcloud site work. Apparently the nextcloud-le-ssl.conf created by certbot didn’t get picked up.


ServerAdmin webmaster@localhost

	DocumentRoot /var/www/html

	# Available loglevels: trace8, ..., trace1, debug, info, notice,

warn,
# error, crit, alert, emerg.
# It is also possible to configure the loglevel for particular
# modules, e.g.
#LogLevel info ssl:warn

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined

	# For most configuration files from conf-available/, which are
	# enabled or disabled at a global level, it is possible to
	# include a line for only one particular virtual host. For examp

le the
# following line enables the CGI configuration for this host onl
y
# after it has been globally disabled with “a2disconf”.
#Include conf-available/serve-cgi-bin.conf

	#   SSL Engine Switch:
	#   Enable/Disable SSL for this virtual host.
	SSLEngine on

	#   A self-signed (snakeoil) certificate can be created by insta

lling
# the ssl-cert package. See
# /usr/share/doc/apache2/README.Debian.gz for more info.
# If both key and certificate are stored in the same file, onl
y the
# SSLCertificateFile directive is needed.
#SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
#SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

	Include /etc/letsencrypt/options-ssl-apache.conf
	SSLCertificateFile /etc/letsencrypt/live/mydomain.com/ful

lchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/mydomain.com/
privkey.pem

	#   Server Certificate Chain:
	#   Point SSLCertificateChainFile at a file containing the
	#   concatenation of PEM encoded CA certificates which form the
	#   certificate chain for the server certificate. Alternatively
	#   the referenced file can be the same as SSLCertificateFile
	#   when the CA certificates are directly appended to the server
	#   certificate for convinience.
	#SSLCertificateChainFile /etc/apache2/ssl.crt/server-ca.crt

	#   Certificate Authority (CA):
	#   Set the CA certificate verification path where to find CA
	#   certificates for client authentication or alternatively one
	#   huge file containing all of them (file must be PEM encoded)
	#   Note: Inside SSLCACertificatePath you need hash symlinks
	#		 to point to the certificate files. Use the prov

ided
# Makefile to update the hash symlinks after chan
ges.
#SSLCACertificatePath /etc/ssl/certs/
#SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt

	#   Certificate Revocation Lists (CRL):
	#   Set the CA revocation path where to find CA CRLs for client
	#   authentication or alternatively one huge file containing all
	#   of them (file must be PEM encoded)
	#   Note: Inside SSLCARevocationPath you need hash symlinks
	#		 to point to the certificate files. Use the prov

ided
# Makefile to update the hash symlinks after chan
ges.
#SSLCARevocationPath /etc/apache2/ssl.crl/
#SSLCARevocationFile /etc/apache2/ssl.crl/ca-bundle.crl

	#   Client Authentication (Type):
	#   Client certificate verification type and depth.  Types are
	#   none, optional, require and optional_no_ca.  Depth is a
	#   number which specifies how deeply to verify the certificate
	#   issuer chain before deciding the certificate is not valid.
	#SSLVerifyClient require
	#SSLVerifyDepth  10

	#   SSL Engine Options:
	#   Set various options for the SSL engine.
	#   o FakeBasicAuth:
	#	 Translate the client X.509 into a Basic Authorisation. 

This means that
# the standard Auth/DBMAuth methods can be used for acces
s control. The
# user name is the one line' version of the client's X.5 09 certificate. # Note that no password is obtained from the user. Every entry in the user # file needs this password: xxj31ZMTZzkVA’.
# o ExportCertData:
# This exports two additional environment variables: SSL_
CLIENT_CERT and
# SSL_SERVER_CERT. These contain the PEM-encoded certific
ates of the
# server (always existing) and the client (only existing
when client
# authentication is used). This can be used to import the
certificates
# into CGI scripts.
# o StdEnvVars:
# This exports the standard SSL/TLS related `SSL_*’ envir
onment variables.
# Per default this exportation is switched off for perfor
mance reasons,
# because the extraction step is an expensive operation a
nd is usually
# useless for serving static content. So one usually enab
les the
# exportation for CGI and SSI requests only.
# o OptRenegotiate:
# This enables optimized SSL connection renegotiation han
dling when SSL
# directives are used in per-directory context.
#SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
<FilesMatch “.(cgi|shtml|phtml|php)$”>
SSLOptions +StdEnvVars

<Directory /usr/lib/cgi-bin>
SSLOptions +StdEnvVars

	#   SSL Protocol Adjustments:
	#   The safe and default but still SSL/TLS standard compliant sh

utdown
# approach is that mod_ssl sends the close notify alert but do
esn’t wait for
# the close notify alert from client. When you need a differen
t shutdown
# approach you can use one of the following variables:
# o ssl-unclean-shutdown:
# This forces an unclean shutdown when the connection is
closed, i.e. no
# SSL close notify alert is send or allowed to received.
This violates
# the SSL/TLS standard but is needed for some brain-dead
browsers. Use
# this when you receive I/O errors because of the standar
d approach where
# mod_ssl sends the close notify alert.
# o ssl-accurate-shutdown:
# This forces an accurate shutdown when the connection is
closed, i.e. a
# SSL close notify alert is send and mod_ssl waits for th
e close notify
# alert of the client. This is 100% SSL/TLS standard comp
liant, but in
# practice often causes hanging connections with brain-de
ad browsers. Use
# this only for browsers where you know that their SSL im
plementation
# works correctly.
# Notice: Most problems of broken clients are also related to
the HTTP
# keep-alive facility, so you usually additionally want to dis
able
# keep-alive for those clients, too. Use variable "nokeepalive
" for this.
# Similarly, one has to force some clients to use HTTP/1.0 to
workaround
# their broken HTTP/1.1 implementation. Use variables “downgra
de-1.0” and
# “force-response-1.0” for this.
# BrowserMatch “MSIE [2-6]”
# nokeepalive ssl-unclean-shutdown
# downgrade-1.0 force-response-1.0

</VirtualHost>

vim: syntax=apache ts=4 sw=4 sts=4 sr noet

1 Like

Thanks. I tried installing Nextcloud on Ubuntu Focal using the VM scripts (https://github.com/nextcloud/vm/)

Unfortunately it automatically setup Certbot and created a certificate for me, so I couldn’t reproduce your problem. It appears to use this command:

certbot --standalone -d example.org --pre-hook "systemctl stop apache2.service" --post-hook "systemctl start apache2.service"

So it’s quite possible that Nextcloud know that certbot --apache doesn’t work with the Apache configuration they use.

Are you able to link me to what instructions you used to setup Nextcloud? I was hoping to end up with the same Apache configuration as you, but I got a completely different one.

1 Like

Thanks for looking into this! I installed nextcloud from scratch by following this:

https://docs.nextcloud.com/server/latest/admin_manual/installation/source_installation.html

1 Like

does list Apache rewrite rules as potentially Turing-complete, citing

http://olsner.se/2008/01/21/an-excursion-in-mod_rewrite/

2 Likes

Thanks. Using those instructions, I didn’t encounter any issues either. Grrr. These are tricky to chase down. I wish (in general) we had some better way to share full Apache configurations.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.