Failed challenge Ubuntu 20, Apache 2, Django

Help
1

I am trying to migrate from Ubuntu 18 -> 20, as well as django old -> current, and python2 -> 3. It seems that my new server's configuration is broken somehow and I haven't been able to get certbot to succeed despite several hours of trial & error, google/stackoverflow, and searching this forum. I would really appreciate some help!

I'm trying to include all the information I've seen requested in other posts below; obviously I'm happy to provide more if I'm leaving something out.

It seems a lot like django is gobbling up requests that I don't want it to, and causing my HTTP-01 challenge to fail at http://bailinphil.com/.well-known/acme-challenge/MbpIAWbRlQ7EbaqZxl5E5jaGTJ_Kqzvl9n_DqcBUhI8 . But this is just my working theory.

My domain is:

I ran this command:

sudo certbot --apache

It produced this output:
...
- The following errors were reported by the server:

 Domain: bailinphil.com
 Type:   unauthorized
 Detail: Invalid response from
 http://bailinphil.com/.well-known/acme-challenge/MbpIAWbRlQ7EbaqZxl5E5jaGTJ_Kqzvl9n_DqcBUhI8
 [161.35.100.35]: "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n
 <meta http-equiv=\"content-type\" content=\"text/html;
 charset=utf-8\">\n  <title>Page not "

 Domain: electricowlstudios.com
 Type:   unauthorized
 Detail: Invalid response from
 http://electricowlstudios.com/.well-known/acme-challenge/Y86m-LoA9x1ZmCEUEQuHWVrwV05zrQZLUdmsIn9eaXk
 [161.35.100.35]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
 2.0//EN\">\n<html><head>\n<title>404 Not
 Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

 Domain: www.bailinphil.com
 Type:   unauthorized
 Detail: Invalid response from
 http://www.bailinphil.com/.well-known/acme-challenge/hg8DZnJJEpLGDajhnuk9XSRkkuePohYUzsIexrrAOcg
 [161.35.100.35]: "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n
 <meta http-equiv=\"content-type\" content=\"text/html;
 charset=utf-8\">\n  <title>Page not "

 Domain: www.electricowlstudios.com
 Type:   unauthorized
 Detail: Invalid response from
 http://www.electricowlstudios.com/.well-known/acme-challenge/ntrSyE6CR3Rd5gfulYKGss48Rm1YjxkWC4EMzdCE1nA
 [161.35.100.35]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
 2.0//EN\">\n<html><head>\n<title>404 Not
 Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"
...

My web server is (include version):

Apache/2.4.41 (Ubuntu)

The operating system my web server runs on is (include version):

Ubuntu 20.04.2 LTS

My hosting provider is:

Digital Ocean

I can login to a root shell on my machine

yes

I'm using a control panel to manage my site

no

The version of my client is

certbot 1.12.0

apachectl -S
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
VirtualHost configuration:
*:80 is a NameVirtualHost
default server bailinphil.com (/etc/apache2/sites-enabled/bailinsite.conf:2)
port 80 namevhost bailinphil.com (/etc/apache2/sites-enabled/bailinsite.conf:2)
alias www.bailinphil.com
port 80 namevhost electricowlstudios.com (/etc/apache2/sites-enabled/electricowl.conf:1)
alias www.electricowlstudios.com
ServerRoot: "/etc/apache2"
Main DocumentRoot: "/var/www/html"
Main ErrorLog: "/var/log/apache2/error.log"
Mutex watchdog-callback: using_defaults
Mutex rewrite-map: using_defaults
Mutex default: dir="/var/run/apache2/" mechanism=default
PidFile: "/var/run/apache2/apache2.pid"
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="www-data" id=33 not_used
Group: name="www-data" id=33 not_used

Apache configuration

<VirtualHost *:80>
  ServerName bailinphil.com
  ServerAlias www.bailinphil.com
  ServerAdmin webmaster@localhost
  Alias /robots.txt /mnt/backlog_photos_db/var/website/robots.txt
  Alias /favicon.ico /mnt/backlog_photos_db/var/website/static/images/favicon.ico
  Alias /images/ /mnt/backlog_photos_db/var/website/static/images/
  Alias /about/ /mnt/backlog_photos_db/var/website/static/about/
  Alias /css/ /mnt/backlog_photos_db/var/website/static/css/
  Alias /static/ /mnt/backlog_photos_db/var/website/static/
  <Directory /mnt/backlog_photos_db/var/website/static>
    Options -Indexes +FollowSymLinks +MultiViews
    AllowOverride None
    Order allow,deny
    allow from all
  </Directory>

  WSGIDaemonProcess mainPyProc python-path=.:/mnt/backlog_photos_db/var/website/bailinsite/backlog/
  WSGIApplicationGroup %{GLOBAL}

  WSGIScriptAlias / /mnt/backlog_photos_db/var/website/bailinsite/bailinsite/wsgi.py process-group=mainPyProc

  ErrorLog ${APACHE_LOG_DIR}/bailinsite-error.log

  # Possible values include: debug, info, notice, warn, error, crit,
  # alert, emerg.
  LogLevel debug

  CustomLog ${APACHE_LOG_DIR}/access.log combined

RewriteEngine off
RewriteCond %{SERVER_NAME} =bailinphil.com [OR]
RewriteCond %{SERVER_NAME} =www.bailinphil.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

Additional debugging

I see on https://check-your-website.server-daten.de/?q=bailinphil.com that I should add an AAAA record for IPv6. I'm working on that now, but don't see anything else in the results which seem useful.

Thank you!

2

Hi @plight

there is a Django answer.

I don't know how Django works. But I don't think that can work with --apache.

  • Ask in a Django forum

or:

Check, how Django can handle static files in subdirectories.

Then define one in /.well-known/acme-challenge and switch to --webroot.

That's not relevant. Ipv6 is good. But if you have a working ipv4 without an ipv6, that should work.

That's independend from your Django configuration.

3

Thank you! I'll keep searching for Django & Apache sites. This same setup worked the first time on my previous server (Ubuntu 18, etc) so I think that it should still be possible. I'll keep digging.

From my earlier reading about --webroot, it looked like this is something I would need to do manually every 60-90 days?

It's pretty straightforward for me to serve a static file and preempt django (as I'm doing for things like robots.txt in the example above.) But I'm not sure exactly what I should be putting in /.well-known/acme-challenge... will the challenge file be returned when I run with --webroot? (I'll go try to search the docs.)

Thanks again!

1 Like
4

Certbot will do that if you use webroot.

But your Django must send that static file -> that's what you have to configure, so it's a Django configuration problem.

5

I was able to solve this, thank you so much! I'm leaving notes here for anyone else who searches for this:

  1. Added the following to my apache configuration in sites-available/

    Alias /.well-known/ /var/www/html/.well-known/
    <Directory /var/www/html/.well-known>
    Options -Indexes +FollowSymLinks +MultiViews
    AllowOverride None
    Order allow,deny
    allow from all

  2. Run the certonly command for "domain only" and "www" versions (I tried the "specify multiple domains" option as indicated in the docs but it didn't seem to work for me. Probably my mistake.)
    sudo certbot certonly --webroot -w /var/www/html -d bailinphil.com
    sudo certbot certonly --webroot -w /var/www/html -d www.bailinphil.com

This succeeded and I was able to generate the certificates.

  1. I had a dated version of the SSL site configuration still floating around in /etc/apache2/sites-available, and I had to modify it to:
  • have the same set of aliases in both the SSL and unencrypted configuration
  • define my wsgi python instance in /etc/apache2/apache2.conf, migrating these 2 lines from the site configuration to the shared file:
    WSGIDaemonProcess mainPyProc python-path=.:/mnt/backlog_photos_db/var/website/bailinsite/backlog/
    WSGIApplicationGroup %{GLOBAL}
  • remove those two lines from the sites-available/ configuration files, but continue to refer to them in the line which tells Apache to process most URLs with Django:
    WSGIScriptAlias / /mnt/backlog_photos_db/var/website/bailinsite/bailinsite/wsgi.py process-group=mainPyProc

  1. re-generate the SSL versions of the configuration with:
    sudo certbot --apache

  2. Confirm that the HTTPS version of my site is working

  3. Turn the RewriteEngine on (in my non-encrypted conf file)

It's really surprising to me that everything was so complex, since the first time I installed the Let's Encrypt certificates on the previous version of this server, all I remember doing was "certbot --apache" and everything just worked. It sure seems that I got my configuration all messed up.

The last step I need to do is make sure that the automated renewal is set up; "systemctl list-timers" seems to indicate that it is.

Thanks again!

6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.