Folder ".well-known/acme-challenge" disappeared

Sherratt · July 27, 2020, 1:27pm

I’m relatively ignorant about how Certbot works. However, I have had a server with a Certbot cert for a year or so. Cron job set up to renew, and normally it renews 30 days before expiry with no problem. The cron job ran early this morning and got a 404 on /.well-known/acme-challenge! I’ve looked around my file system and cannot find a .well-known folder. I don’t see an Alias in the Apache config files.

I did revoke a test certificate a month or so ago, and it is possible that I touched something at that time that messed things up… Too long ago to remember what else I did at the same time.

I’ve looked at other queries on community pages, but nothing seems similar enough. Should I see a .well-known folder? Any ideas where it would be? If it isn’t there, is there some way to recreate it? The website would need to continue functioning throughout.
Thanks!

My domain is: bslbeam.co.uk

I ran this command:
sudo certbot renew

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/bslbeam.co.uk.conf

Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator apache, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for www.bslbeam.com
Waiting for verification…
Cleaning up challenges
Attempting to renew cert (bslbeam.co.uk) from /etc/letsencrypt/renewal/bslbeam.co.uk.conf produced an unexpected error: Failed authorization procedure. www.bslbeam.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from https://bslbeam.co.uk/.well-known/acme-challenge/sulXjyxMyamZDnIqwe5RSW1Rwg77TL3zT_OoRtMMRVY [109.228.57.245]: “\r\n<html xmlns=“http://www.w3.org/1999/xhtml” xml:lang=“en-gb” lang=“en-gb”>\r\n \r\n Language Beam</”. Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/bslbeam.co.uk/fullchain.pem (failure)

All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/bslbeam.co.uk/fullchain.pem (failure)

1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

The following errors were reported by the server:

Domain: www.bslbeam.com
Type: unauthorized
Detail: Invalid response from
https://bslbeam.co.uk/.well-known/acme-challenge/sulXjyxMyamZDnIqwe5RSW1Rwg77TL3zT_OoRtMMRVY
[109.228.57.245]: “\r\n<html
xmlns=“http://www.w3.org/1999/xhtml” xml:lang=“en-gb”
lang=“en-gb”>\r\n \r\n Language Beam</”

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

My web server is (include version):
Apache/2.4.7 (Ubuntu)

The operating system my web server runs on is (include version):
Ubuntu 14.04.6 LTS

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don’t know):
Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):
certbot 0.31.0

JuergenAuer · July 27, 2020, 1:31pm

Hi @Sherratt

if you use the Apache authenticator, Letsencrypt shouldn't see a redirect to https.

Looks like your configuration is buggy.

What says

apachectl -S

Remove all duplicated definitions, every combination of port and domain name must be unique.

Sherratt · July 27, 2020, 1:37pm

Thanks Juergen. It definitely looks like it is buggy - see below! I'll dig into the conf file

AH00526: Syntax error on line 367 of /etc/apache2/sites-enabled/bslbeam-secure.conf:
SSLCertificateFile: file '/etc/letsencrypt/live/bslbeam.co.uk/fullchain.pem' does not exist or is empty
Action '-S' failed.
The Apache error log may have more information.

Sherratt · July 27, 2020, 2:14pm

Looked at config files and there are only 2 virtual hosts defined, one on port 80 and the other on 443.

However, the *:80 virtual host rewrites to https - wouldn't that redirect Letsencrypt to https? In any case, it has been like that for years, and hasn't blocked the cert renewal since I changed over to Letsencrypt

The cert file that apachectl says does not exist, is where it is supposed to be and isn't empty! I checked the symlink is pointing to the most recent cert file, and
cat /etc/letsencrypt/live/bslbeam.co.uk/fullchain.pem
looks like a normal cert.

I wondered whether there might be a syntax error in the conf file prior to line 367, but I can't see any problems.

What about the ".well-known" directory? Should it exist?

JuergenAuer · July 27, 2020, 2:44pm

Run it as root or sudo.

As written: If you use --apache, another definition is included, so that should be skipped. But that doesn't work, may be that vHost isn't used or your configuration is too complex. Perhaps use the webroot of the https version with --webroot.

https://certbot.eff.org/docs/using.html

Sherratt · July 27, 2020, 2:52pm

Doh! Thought I had. Sorry.
VirtualHost configuration:
*:443 bslbeam.co.uk (/etc/apache2/sites-enabled/bslbeam-secure.conf:2)
*:80 bslbeam.co.uk (/etc/apache2/sites-enabled/bslbeam.conf:1)
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex watchdog-callback: using_defaults
Mutex rewrite-map: using_defaults
Mutex ssl-stapling: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/var/lock/apache2" mechanism=fcntl
Mutex mpm-accept: using_defaults
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
Define: MODPERL2
Define: ENABLE_USR_LIB_CGI_BIN
User: name="www-data" id=33
Group: name="www-data" id=33

9peppe · July 27, 2020, 3:42pm

You should upgrade this machine.

are port 80 and 443 sharing a DocumentRoot? I suspect your redirect is too aggressive and breaks http-01 verification. Can you show us the files in /etc/apache2/sites-enabled/?

Sherratt · July 28, 2020, 9:45am

Yep, its in plan. Although I had thought it was 16.04.6 !

Sherratt · July 28, 2020, 9:48am

Does anyone know whether there should be a “.well-known” folder?

9peppe · July 28, 2020, 10:48am

Usually, there shouldn’t. Certbot will create it if there isn’t one.

But be aware that there could be a .well-known and other stuff, unrelated to acme-challenge, could be in there.

Sherratt · July 28, 2020, 1:11pm

Certbot will create it if there isn’t one.

Thanks. I assume it would have been created at the time Certbot was first run. Given that it wasn't getting 404s before, and now it is, I guess .well-known has somehow been deleted!

9peppe · July 28, 2020, 1:59pm

If certbot creates it, it will remove it after. But that depends on your authenticator plugin: are you using --apache or --webroot?

Sherratt · July 29, 2020, 4:47pm

Plugins selected: Authenticator apache, Installer None

From memory, when it was first set up, there were some glitches when trying with --apache, so may have ended up using --webroot. However, whichever was used, the Cron job renewal has gone through every time until now, so I'm surprised it is now getting a 404.

Maybe better to manually try cert-only?

9peppe · July 29, 2020, 5:06pm

No, just show us what your apache configuration looks like.

schoen · July 29, 2020, 5:55pm

By the way, you might already be aware of this, but --webroot uses a /.well-known/acme-challenge in your existing webroot, while --apache doesn't (it makes its own path and modifies your Apache configuration to point at it, then modifies your Apache configuration back after the authentication process is complete).

Sherratt · August 2, 2020, 1:23pm

Sorry about the delay. Other things have got in the way. Still looking for a solution to this!

Thanks schoen. I wasn't aware of that. In which case the answer to a previous question should have been I am using --webroot.

The config file gets updated fairly frequently and is probably more complicated than it needs to be! Overnight I put up a much simplified config file taking out the layers of redirects that are usually there. The renew command still came back saying it had hit a 404.

There are 2 active config files. Any ideas?

<VirtualHost *:80>
  ServerName bslbeam.co.uk
  ServerAlias languagebeam.com
  ServerAlias languagebeam.co.uk
  ServerAlias bslbeam.com
  ServerAlias www.bslbeam.co.uk
  ServerAlias www.languagebeam.com
  ServerAlias www.languagebeam.co.uk
  ServerAlias www.bslbeam.com

  ServerAdmin webmaster@localhost
  
  RewriteEngine on
  RewriteRule ^ — [E=protossl]
  RewriteCond %{HTTPS} on
  RewriteRule ^ — [E=protossl:s]

  # Rewrite http(s)://example.com to https://example.com
  #
  RewriteCond “%{HTTP_HOST}” “!^www\.” [NC]
  RewriteCond “%{HTTP_HOST}” “!^$”
  RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

  #
  # Rewrite http://www.example.com to https://www.example.com
  #
  RewriteCond %{HTTPS} off
  RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
  
</VirtualHost>

And

  <IfModule mod_ssl.c>
  <VirtualHost _default_:443>
    ServerAdmin admin@bslbeam.co.uk
    ServerName bslbeam.co.uk
    RewriteEngine on
    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn
    #LogLevel alert rewrite:trace8

    DocumentRoot /var/www/versionCurrent
    Alias /current /var/www/versionCurrent

    #Redirect bslbeam.com
    RewriteCond "%{HTTP_HOST}" "^(www\.)?bslbeam\.com"
    RewriteRule ".*" "https://bslbeam.co.uk" [R=302,L]

    #Redirect to shared folders
    AliasMatch "^/?[^/]*?/?(flaticons|files|images|icons32|icons16|fpdi)/(.*)$" "/var/www/versionsALL/$1/$2"

    Alias /2108 /var/www/version2_10_8
    Alias /2107 /var/www/version2_10_7
    Alias /2106 /var/www/version2_10_6
    Alias /2105 /var/www/version2_10_5
    Alias /2104 /var/www/version2_10_4
    Alias /2103 /var/www/version2_10_3
    Alias /2102 /var/www/version2_10_2
    Alias /2101 /var/www/version2_10_1
    Alias /2100 /var/www/version2_10_0

    Alias /298 /var/www/version2_9_8
    Alias /297 /var/www/version2_9_7
    Alias /296 /var/www/version2_9_6
    Alias /295 /var/www/version2_9_5
    Alias /294 /var/www/version2_9_4
    Alias /293 /var/www/version2_9_3
    Alias /292 /var/www/version2_9_2
    Alias /291 /var/www/version2_9_1
    Alias /290 /var/www/version2_9_0

    Alias /beta /var/www/versionBeta
    Alias /alpha /var/www/versionAlpha
    Alias /old /var/www/versionOld
    Alias /all /var/www/versionsALL
    Alias /alphaTestingOver /var/www/alphaTestingOver
    Alias /betaTestingOver /var/www/betaTestingOver

    <Directory />
      Options -Indexes +FollowSymLinks
      AllowOverride None
    </Directory>

    <Directory /var/www/versionCurrent>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride all
      Order allow,deny
      allow from all
    </Directory>

    <Directory /var/www/versionAlpha>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride all
      Order allow,deny
      allow from all
    </Directory>

    <Directory /var/www/versionBeta>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride all
      Order allow,deny
      allow from all
    </Directory>

    <Directory /var/www/versionOld>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride all
      Order allow,deny
      allow from all
    </Directory>

    <Directory /var/www/version2_10_8>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "2107/$1"
    </Directory>

    <Directory /var/www/version2_10_7>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "2106/$1"
    </Directory>

    <Directory /var/www/version2_10_6>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "2105/$1"
    </Directory>

    <Directory /var/www/version2_10_5>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "2104/$1"
    </Directory>

    <Directory /var/www/version2_10_4>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "2103/$1"
    </Directory>

    <Directory /var/www/version2_10_3>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "2102/$1"
    </Directory>

    <Directory /var/www/version2_10_2>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "2101/$1"
    </Directory>

    <Directory /var/www/version2_10_1>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "2100/$1"
    </Directory>

    <Directory /var/www/version2_10_0>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all
    </Directory>

    <Directory /var/www/version2_9_8>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "297/$1"
    </Directory>

    <Directory /var/www/version2_9_7>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "296/$1"
    </Directory>

    <Directory /var/www/version2_9_6>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "295/$1"
    </Directory>

    <Directory /var/www/version2_9_5>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "294/$1"
    </Directory>

    <Directory /var/www/version2_9_4>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "293/$1"
    </Directory>

    <Directory /var/www/version2_9_3>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "292/$1"
    </Directory>

    <Directory /var/www/version2_9_2>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "291/$1"
    </Directory>

    <Directory /var/www/version2_9_1>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all

      RewriteBase "/"
      RewriteCond %{REQUEST_FILENAME} !-f
      RewriteCond %{REQUEST_FILENAME} !-d
      RewriteRule "^(.*)$" "290/$1"
    </Directory>

    <Directory /var/www/version2_9_0>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all
    </Directory>

    <Directory /var/www/alphaTestingOver>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride All
      Order allow,deny
      allow from all
    </Directory>

    <Directory /var/www/betaTestingOver>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride All
      Order allow,deny
      allow from all
    </Directory>

    Alias /bugs /var/www/html/bugzilla
    <Directory /var/www/html/bugzilla>
      AddHandler cgi-script .cgi
      Options +ExecCGI
      DirectoryIndex index.cgi index.html
      AllowOverride Limit FileInfo Indexes Options
    </Directory>

    <Directory /var/www/versionsALL/images>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all
    </Directory>
    <Directory /var/www/versionsALL/icons32>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all
    </Directory>
    <Directory /var/www/versionsALL/icons16>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all
    </Directory>
    <Directory /var/www/versionsALL/files>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all
    </Directory>
    <Directory /var/www/version2_1/fpdi>
      Options -Indexes +FollowSymLinks +MultiViews
      AllowOverride None
      Order allow,deny
      allow from all
    </Directory>

    ErrorLog /var/log/apache2/error.log
    CustomLog /var/log/apache2/ssl_access.log combined

    Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
      Options Indexes MultiViews FollowSymLinks
      AllowOverride None
      Order deny,allow
      Deny from all
      Allow from 127.0.0.0/255.0.0.0 ::1/128
    </Directory>

    #   SSL Engine Switch:
    #   Enable/Disable SSL for this virtual host.
    SSLEngine on

    #   A self-signed (snakeoil) certificate can be created by installing
    #   the ssl-cert package. See
    #   /usr/share/doc/apache2.2-common/README.Debian.gz for more info.
    #   If both key and certificate are stored in the same file, only the
    #   SSLCertificateFile directive is needed.
  #	SSLCertificateFile    /etc/apache2/ssl/bslbeam.crt
  #	SSLCertificateKeyFile /etc/apache2/ssl/bslbeam.key
  #	SSLCertificateFile    /etc/apache2/ssl/8dcf9c560abe266c.crt
  #	SSLCertificateKeyFile /etc/apache2/ssl/bslbeam.key
  #	SSLCertificateFile    /etc/apache2/ssl/35fba7efeb8d9fa5.crt
  #	SSLCertificateKeyFile /etc/apache2/ssl/bslbeam.key
    SSLCertificateFile    /etc/letsencrypt/live/bslbeam.co.uk/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/bslbeam.co.uk/privkey.pem

    #   Server Certificate Chain:
    #   Point SSLCertificateChainFile at a file containing the
    #   concatenation of PEM encoded CA certificates which form the
    #   certificate chain for the server certificate. Alternatively
    #   the referenced file can be the same as SSLCertificateFile
    #   when the CA certificates are directly appended to the server
    #   certificate for convinience.
  #  Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateChainFile /etc/letsencrypt/live/bslbeam.co.uk/chain.pem

    #   Certificate Authority (CA):
    #   Set the CA certificate verification path where to find CA
    #   certificates for client authentication or alternatively one
    #   huge file containing all of them (file must be PEM encoded)
    #   Note: Inside SSLCACertificatePath you need hash symlinks
    #         to point to the certificate files. Use the provided
    #         Makefile to update the hash symlinks after changes.
    #SSLCACertificatePath /etc/ssl/certs/
    #SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt

    #   Certificate Revocation Lists (CRL):
    #   Set the CA revocation path where to find CA CRLs for client
    #   authentication or alternatively one huge file containing all
    #   of them (file must be PEM encoded)
    #   Note: Inside SSLCARevocationPath you need hash symlinks
    #         to point to the certificate files. Use the provided
    #         Makefile to update the hash symlinks after changes.
    #SSLCARevocationPath /etc/apache2/ssl.crl/
    #SSLCARevocationFile /etc/apache2/ssl.crl/ca-bundle.crl

    #   Client Authentication (Type):
    #   Client certificate verification type and depth.  Types are
    #   none, optional, require and optional_no_ca.  Depth is a
    #   number which specifies how deeply to verify the certificate
    #   issuer chain before deciding the certificate is not valid.
    #SSLVerifyClient require
    #SSLVerifyDepth  10

    #   Access Control:
    #   With SSLRequire you can do per-directory access control based
    #   on arbitrary complex boolean expressions containing server
    #   variable checks and other lookup directives.  The syntax is a
    #   mixture between C and Perl.  See the mod_ssl documentation
    #   for more details.
    #<Location />
    #SSLRequire (    %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
    #            and %{SSL_CLIENT_S_DN_O} eq "Snake Oil, Ltd." \
    #            and %{SSL_CLIENT_S_DN_OU} in {"Staff", "CA", "Dev"} \
    #            and %{TIME_WDAY} >= 1 and %{TIME_WDAY} <= 5 \
    #            and %{TIME_HOUR} >= 8 and %{TIME_HOUR} <= 20       ) \
    #           or %{REMOTE_ADDR} =~ m/^192\.76\.162\.[0-9]+$/
    #</Location>

    #   SSL Engine Options:
    #   Set various options for the SSL engine.
    #   o FakeBasicAuth:
    #     Translate the client X.509 into a Basic Authorisation.  This means that
    #     the standard Auth/DBMAuth methods can be used for access control.  The
    #     user name is the `one line' version of the client's X.509 certificate.
    #     Note that no password is obtained from the user. Every entry in the user
    #     file needs this password: `xxj31ZMTZzkVA'.
    #   o ExportCertData:
    #     This exports two additional environment variables: SSL_CLIENT_CERT and
    #     SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
    #     server (always existing) and the client (only existing when client
    #     authentication is used). This can be used to import the certificates
    #     into CGI scripts.
    #   o StdEnvVars:
    #     This exports the standard SSL/TLS related `SSL_*' environment variables.
    #     Per default this exportation is switched off for performance reasons,
    #     because the extraction step is an expensive operation and is usually
    #     useless for serving static content. So one usually enables the
    #     exportation for CGI and SSI requests only.
    #   o StrictRequire:
    #     This denies access when "SSLRequireSSL" or "SSLRequire" applied even
    #     under a "Satisfy any" situation, i.e. when it applies access is denied
    #     and no other module can change it.
    #   o OptRenegotiate:
    #     This enables optimized SSL connection renegotiation handling when SSL
    #     directives are used in per-directory context.
    #SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
    <FilesMatch "\.(cgi|shtml|phtml|php)$">
            SSLOptions +StdEnvVars
    </FilesMatch>
    <Directory /usr/lib/cgi-bin>
            SSLOptions +StdEnvVars
    </Directory>

    #   SSL Protocol Adjustments:
    #   The safe and default but still SSL/TLS standard compliant shutdown
    #   approach is that mod_ssl sends the close notify alert but doesn't wait for
    #   the close notify alert from client. When you need a different shutdown
    #   approach you can use one of the following variables:
    #   o ssl-unclean-shutdown:
    #     This forces an unclean shutdown when the connection is closed, i.e. no
    #     SSL close notify alert is send or allowed to received.  This violates
    #     the SSL/TLS standard but is needed for some brain-dead browsers. Use
    #     this when you receive I/O errors because of the standard approach where
    #     mod_ssl sends the close notify alert.
    #   o ssl-accurate-shutdown:
    #     This forces an accurate shutdown when the connection is closed, i.e. a
    #     SSL close notify alert is send and mod_ssl waits for the close notify
    #     alert of the client. This is 100% SSL/TLS standard compliant, but in
    #     practice often causes hanging connections with brain-dead browsers. Use
    #     this only for browsers where you know that their SSL implementation
    #     works correctly.
    #   Notice: Most problems of broken clients are also related to the HTTP
    #   keep-alive facility, so you usually additionally want to disable
    #   keep-alive for those clients, too. Use variable "nokeepalive" for this.
    #   Similarly, one has to force some clients to use HTTP/1.0 to workaround
    #   their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
    #   "force-response-1.0" for this.
    BrowserMatch "MSIE [2-6]" \
      nokeepalive ssl-unclean-shutdown \
      downgrade-1.0 force-response-1.0
    # MSIE 7 and newer should be able to use keepalive
    BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown

  ErrorDocument 400 /errorPage.php
  ErrorDocument 401 /errorPage.php
  ErrorDocument 403 /errorPage.php
  ErrorDocument 404 /index.php
  ErrorDocument 405 /errorPage.php
  ErrorDocument 408 /errorPage.php
  ErrorDocument 410 /errorPage.php
  ErrorDocument 411 /errorPage.php
  ErrorDocument 412 /errorPage.php
  ErrorDocument 413 /errorPage.php
  ErrorDocument 414 /errorPage.php
  ErrorDocument 415 /errorPage.php
  ErrorDocument 500 /errorPage.php

  </VirtualHost>
  </IfModule>

9peppe · August 2, 2020, 1:31pm

That’s still a lot of rewrite rules and aliases… some might interfere with the authenticator.

Sherratt · August 2, 2020, 1:53pm

Yes, that was the original config file. The simplified one which I only used temporarily as it would not currently work for the live site is:

<IfModule mod_ssl.c>
<VirtualHost _default_:443>  ServerName bslbeam.co.uk
  ServerAlias languagebeam.com
  ServerAlias languagebeam.co.uk
  ServerAlias bslbeam.com
  ServerAlias www.bslbeam.co.uk
  ServerAlias www.languagebeam.com
  ServerAlias www.languagebeam.co.uk
  ServerAlias www.bslbeam.com

  ServerAdmin webmaster@localhost

  DocumentRoot /var/www/beam

  <Directory />
    Options -Indexes +FollowSymLinks
    AllowOverride None
  </Directory>

  <Directory /var/www/beam>
    Options -Indexes +FollowSymLinks +MultiViews
    AllowOverride all
    Order allow,deny
    allow from all
    Require all granted
  </Directory>  

  ErrorLog /var/log/apache2/error.log
  CustomLog /var/log/apache2/ssl_access.log combined

  #   SSL Engine Switch:
  #   Enable/Disable SSL for this virtual host.
  SSLEngine on

  #   A self-signed (snakeoil) certificate can be created by installing
  #   the ssl-cert package. See
  #   /usr/share/doc/apache2.2-common/README.Debian.gz for more info.
  #   If both key and certificate are stored in the same file, only the
  #   SSLCertificateFile directive is needed.
#	SSLCertificateFile    /etc/apache2/ssl/bslbeam.crt
#	SSLCertificateKeyFile /etc/apache2/ssl/bslbeam.key
#	SSLCertificateFile    /etc/apache2/ssl/8dcf9c560abe266c.crt
#	SSLCertificateKeyFile /etc/apache2/ssl/bslbeam.key
#	SSLCertificateFile    /etc/apache2/ssl/35fba7efeb8d9fa5.crt
#	SSLCertificateKeyFile /etc/apache2/ssl/bslbeam.key
  SSLCertificateFile    /etc/letsencrypt/live/bslbeam.co.uk/fullchain.pem
  SSLCertificateKeyFile /etc/letsencrypt/live/bslbeam.co.uk/privkey.pem

  #   Server Certificate Chain:
  #   Point SSLCertificateChainFile at a file containing the
  #   concatenation of PEM encoded CA certificates which form the
  #   certificate chain for the server certificate. Alternatively
  #   the referenced file can be the same as SSLCertificateFile
  #   when the CA certificates are directly appended to the server
  #   certificate for convinience.
#  Include /etc/letsencrypt/options-ssl-apache.conf
  SSLCertificateChainFile /etc/letsencrypt/live/bslbeam.co.uk/chain.pem

  #   Certificate Authority (CA):
  #   Set the CA certificate verification path where to find CA
  #   certificates for client authentication or alternatively one
  #   huge file containing all of them (file must be PEM encoded)
  #   Note: Inside SSLCACertificatePath you need hash symlinks
  #         to point to the certificate files. Use the provided
  #         Makefile to update the hash symlinks after changes.
  #SSLCACertificatePath /etc/ssl/certs/
  #SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt

  #   Certificate Revocation Lists (CRL):
  #   Set the CA revocation path where to find CA CRLs for client
  #   authentication or alternatively one huge file containing all
  #   of them (file must be PEM encoded)
  #   Note: Inside SSLCARevocationPath you need hash symlinks
  #         to point to the certificate files. Use the provided
  #         Makefile to update the hash symlinks after changes.
  #SSLCARevocationPath /etc/apache2/ssl.crl/
  #SSLCARevocationFile /etc/apache2/ssl.crl/ca-bundle.crl

  #   Client Authentication (Type):
  #   Client certificate verification type and depth.  Types are
  #   none, optional, require and optional_no_ca.  Depth is a
  #   number which specifies how deeply to verify the certificate
  #   issuer chain before deciding the certificate is not valid.
  #SSLVerifyClient require
  #SSLVerifyDepth  10

  #   Access Control:
  #   With SSLRequire you can do per-directory access control based
  #   on arbitrary complex boolean expressions containing server
  #   variable checks and other lookup directives.  The syntax is a
  #   mixture between C and Perl.  See the mod_ssl documentation
  #   for more details.
  #<Location />
  #SSLRequire (    %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
  #            and %{SSL_CLIENT_S_DN_O} eq "Snake Oil, Ltd." \
  #            and %{SSL_CLIENT_S_DN_OU} in {"Staff", "CA", "Dev"} \
  #            and %{TIME_WDAY} >= 1 and %{TIME_WDAY} <= 5 \
  #            and %{TIME_HOUR} >= 8 and %{TIME_HOUR} <= 20       ) \
  #           or %{REMOTE_ADDR} =~ m/^192\.76\.162\.[0-9]+$/
  #</Location>

  #   SSL Engine Options:
  #   Set various options for the SSL engine.
  #   o FakeBasicAuth:
  #     Translate the client X.509 into a Basic Authorisation.  This means that
  #     the standard Auth/DBMAuth methods can be used for access control.  The
  #     user name is the `one line' version of the client's X.509 certificate.
  #     Note that no password is obtained from the user. Every entry in the user
  #     file needs this password: `xxj31ZMTZzkVA'.
  #   o ExportCertData:
  #     This exports two additional environment variables: SSL_CLIENT_CERT and
  #     SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
  #     server (always existing) and the client (only existing when client
  #     authentication is used). This can be used to import the certificates
  #     into CGI scripts.
  #   o StdEnvVars:
  #     This exports the standard SSL/TLS related `SSL_*' environment variables.
  #     Per default this exportation is switched off for performance reasons,
  #     because the extraction step is an expensive operation and is usually
  #     useless for serving static content. So one usually enables the
  #     exportation for CGI and SSI requests only.
  #   o StrictRequire:
  #     This denies access when "SSLRequireSSL" or "SSLRequire" applied even
  #     under a "Satisfy any" situation, i.e. when it applies access is denied
  #     and no other module can change it.
  #   o OptRenegotiate:
  #     This enables optimized SSL connection renegotiation handling when SSL
  #     directives are used in per-directory context.
  #SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
  <FilesMatch "\.(cgi|shtml|phtml|php)$">
          SSLOptions +StdEnvVars
  </FilesMatch>
  <Directory /usr/lib/cgi-bin>
          SSLOptions +StdEnvVars
  </Directory>

  #   SSL Protocol Adjustments:
  #   The safe and default but still SSL/TLS standard compliant shutdown
  #   approach is that mod_ssl sends the close notify alert but doesn't wait for
  #   the close notify alert from client. When you need a different shutdown
  #   approach you can use one of the following variables:
  #   o ssl-unclean-shutdown:
  #     This forces an unclean shutdown when the connection is closed, i.e. no
  #     SSL close notify alert is send or allowed to received.  This violates
  #     the SSL/TLS standard but is needed for some brain-dead browsers. Use
  #     this when you receive I/O errors because of the standard approach where
  #     mod_ssl sends the close notify alert.
  #   o ssl-accurate-shutdown:
  #     This forces an accurate shutdown when the connection is closed, i.e. a
  #     SSL close notify alert is send and mod_ssl waits for the close notify
  #     alert of the client. This is 100% SSL/TLS standard compliant, but in
  #     practice often causes hanging connections with brain-dead browsers. Use
  #     this only for browsers where you know that their SSL implementation
  #     works correctly.
  #   Notice: Most problems of broken clients are also related to the HTTP
  #   keep-alive facility, so you usually additionally want to disable
  #   keep-alive for those clients, too. Use variable "nokeepalive" for this.
  #   Similarly, one has to force some clients to use HTTP/1.0 to workaround
  #   their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
  #   "force-response-1.0" for this.
  BrowserMatch "MSIE [2-6]" \
    nokeepalive ssl-unclean-shutdown \
    downgrade-1.0 force-response-1.0
  # MSIE 7 and newer should be able to use keepalive
  BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown


</VirtualHost>
</IfModule>

9peppe · August 2, 2020, 2:09pm

Your website works, right?

Try with certbot renew --dry-run --webroot --webroot-path /var/www/beam

Sherratt · August 2, 2020, 3:09pm

Yes, it works. The certificate is still valid for a few more days. The certbot command you suggested still gives an error:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/bslbeam.co.uk.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Attempting to renew cert (bslbeam.co.uk) from /etc/letsencrypt/renewal/bslbeam.co.uk.conf produced an unexpected error: urn:ietf:params:acme:error:malformed :: The request message was malformed :: Method not allowed. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/bslbeam.co.uk/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/bslbeam.co.uk/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)