AWS load balancer + Certbot

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: suhozid.hr

I ran this command: apachectl stop

It produced this output:
AH00526: Syntax error on line 53 of /etc/apache2/sites-enabled/suhozid.hr-le-ssl.conf:
SSLCertificateFile: file '/etc/letsencrypt/live/suhozid.hr-0001/fullchain.pem' does not exist or is empty
Action 'stop' failed.
The Apache error log may have more information.

My web server is (include version): Apache 2.4.52

The operating system my web server runs on is (include version): Ubuntu 22.04

My hosting provider, if applicable, is: AWS

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 2.10.0

upon running: sudo certbot certificates, I got:
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Found the following certs:
Certificate Name: suhozid.hr-0001
Serial Number: 4881982c1e052fe7dcf3e326e8f97bc5b2d
Key Type: ECDSA
Domains: suhozid.hr
Expiry Date: 2024-07-24 12:24:15+00:00 (VALID: 88 days)
Certificate Path: /etc/letsencrypt/live/suhozid.hr-0001/fullchain.pem
Private Key Path: /etc/letsencrypt/live/suhozid.hr-0001/privkey.pem
Certificate Name: suhozid.hr
Serial Number: 37177c633d535a7ce3051632eabe2277538
Key Type: ECDSA
Domains: suhozid.hr www.suhozid.hr
Expiry Date: 2024-07-11 11:56:05+00:00 (VALID: 75 days)
Certificate Path: /etc/letsencrypt/live/suhozid.hr/fullchain.pem
Private Key Path: /etc/letsencrypt/live/suhozid.hr/privkey.pem
Certificate Name: www.suhozid.hr
Serial Number: 3c98780c83d4ba5a8fcb851c4b502c919ef
Key Type: ECDSA
Domains: www.suhozid.hr
Expiry Date: 2024-07-24 12:54:41+00:00 (VALID: 89 days)
Certificate Path: /etc/letsencrypt/live/www.suhozid.hr/fullchain.pem
Private Key Path: /etc/letsencrypt/live/www.suhozid.hr/privkey.pem

the .conf file:

<IfModule mod_ssl.c>
<VirtualHost *:443>

    WSGIApplicationGroup %{GLOBAL}
#    WSGIDaemonProcess arches python-path=/home/ubuntu/suhozid
    WSGIProcessGroup arches
#    WSGIScriptAlias / /home/ubuntu/suhozid/suhozid/wsgi.py process-group=arches

    # May be necessary to support integration with possible 3rd party mobile apps
    WSGIPassAuthorization on

    ## Uncomment the ServerName directive and fill it with your domain
    ## or subdomain if/when you have your DNS records configured.
     ServerName suhozid.hr

    <Directory /home/ubuntu/suhozid/>
        Require all granted
    </Directory>
    # This section tells Apache where to find static files. This example uses
    # STATIC_URL = '/media/' and STATIC_ROOT = os.path.join(APP_ROOT, 'static')
    # NOTE: omit this section if you are using S3 to serve static files.
    Alias /static/ /home/ubuntu/suhozid/suhozid/static/
    <Directory /home/ubuntu/suhozid/suhozid/static/>
        Require all granted
    </Directory>

    # This section tells Apache where to find uploaded files. This example uses
    # MEDIA_URL = '/files/' and MEDIA_ROOT = os.path.join(APP_ROOT)
    # NOTE: omit this section if you are using S3 for uploaded media
    Alias /files/uploadedfiles/ /home/ubuntu/suhozid/suhozid/uploadedfiles/
    <Directory /home/ubuntu/suhozid/suhozid/uploadedfiles/>
        Require all granted
    </Directory>

    ServerAdmin ydrea.wrld@gmail.com
    DocumentRoot /var/www/html

    # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
    # error, crit, alert, emerg.
    # It is also possible to configure the loglevel for particular
    # modules, e.g.
    #LogLevel info ssl:warn
    # Recommend changing these file names if you have multiple arches
    # installations on the same server.
    ErrorLog /var/log/apache2/error-arches.log
    CustomLog /var/log/apache2/access-arches.log combined


ServerName suhozid.hr
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/suhozid.hr-0001/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/suhozid.hr-0001/privkey.pem
</VirtualHost>
</IfModule>

the relevant log bit:

2024-04-12 12:55:57,789:DEBUG:urllib3.connectionpool:Resetting dropped connection: acme-v02.api.letsencrypt.org
2024-04-12 12:55:58,303:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 201
 473
2024-04-12 12:55:58,304:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Date: Fri, 12 Apr 2024 12:55:58 GMT
Content-Type: application/json
Content-Length: 473
Connection: keep-alive
Boulder-Requester: 1668088907
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Location: https://acme-v02.api.letsencrypt.org/acme/order/1668088907/260305912577
Replay-Nonce: 41Xn7Ejc8ygAyqcCYoOWePK69Kodtfiv_F4XXPHqdISCyloXcQc
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
2

Welcome to the community @ydrea

Since Apache is complaining about a missing file would you show output of this

sudo ls -lR /etc/letsencrypt/{live,archive}/suhozid.hr-0001

I see you are using CloudFront and this Apache is behind that. Are you already using HTTPS between CloudFront and Apache? If you could give a brief description of what you are trying to do that would help us instruct you.

3 Likes
3

certainly!


/etc/letsencrypt/archive/suhozid.hr-0001:
total 16
-rw-r--r-- 1 root root 1476 Apr 25 13:24 cert1.pem
-rw-r--r-- 1 root root 1826 Apr 25 13:24 chain1.pem
-rw-r--r-- 1 root root 3302 Apr 25 13:24 fullchain1.pem
-rw------- 1 root root  241 Apr 25 13:24 privkey1.pem

/etc/letsencrypt/live/suhozid.hr-0001:
total 4
-rw-r--r-- 1 root root 692 Apr 25 13:24 README
lrwxrwxrwx 1 root root  39 Apr 25 13:24 cert.pem -> ../../archive/suhozid.hr-0001/cert1.pem
lrwxrwxrwx 1 root root  40 Apr 25 13:24 chain.pem -> ../../archive/suhozid.hr-0001/chain1.pem
lrwxrwxrwx 1 root root  44 Apr 25 13:24 fullchain.pem -> ../../archive/suhozid.hr-0001/fullchain1.pem
lrwxrwxrwx 1 root root  42 Apr 25 13:24 privkey.pem -> ../../archive/suhozid.hr-0001/privkey1.pem

CloudFront, Load Balancer, Target Group and Area 53 are all set up. This is the last piece of the puzzle - trying to make the app server to use HTTPS...

4

Just so I am clear ... Your CloudFront Origin is a Load Balancer? And your Apache sits behind the Load Balancer?

What kind of challenge did you use for the certs? HTTP or DNS? I'm curious as using an HTTP Challenge behind CloudFront and a Load Balancer is complex.

As for your Apache error I don't see why it would complain about the missing file. It is clearly there. And, I see Apache responding to HTTPS requests.

What does this show

sudo apache2ctl -t -D DUMP_VHOSTS
3 Likes
5

if I understand this AWS mess, the load balancer sits behind cloudfront, and in front of the apache :wink:

it was DNS, the domains are parked elsewhere (orbis.hr).


ubuntu@ip-172-31-27-140:~$ sudo apache2ctl -t -D DUMP_VHOSTS
VirtualHost configuration:
*:443                  suhozid.hr (/etc/apache2/sites-enabled/suhozid.hr-le-ssl.conf:3)
*:80                   suhozid.hr (/etc/apache2/sites-enabled/suhozid.hr.conf:7)

weird, right?

6

Yes, especially since it was a STOP that failed. I don't know how you could have started it.

Do you need to use sudo apachectl stop rather than just apachectl stop ?

File might be inaccessible due to permissions

4 Likes
7

The start/stop script probably also checks before stopping, as changes made after stopping Apache could make it defunct and it won't come back up again. As a failsafe.

1 Like
8

whatever I do, all I'm getting is the default apache "It works!" page...

9

I see a home page for your site and not just the default Apache page. I do not see any problems but with a complex server configuration I can't see all the pieces like you can.

CloudFront has gotten a cert from Amazon (not Let's Encrypt) on your behalf for your domain suhozid.hr. You probably want to also include your www subdomain but that's your choice.

Are you still having a problem using Let's Encrypt certs? If so, are those certs for HTTPS between your Apache and your Load Balancer or between Apache and CloudFront? And, what is the current problem?

2 Likes
10

where? how?! I don't see it on any of my browsers, incognito modes on/off...

11

the certbot one is between apache and the load balancer...

12

Cloudfront's network will serve your domain to the public internet, and they will obtain a SSL Certificate from you automatically.

To secure the connection between your system and Cloudfront, you do not need a LetsEncrypt Certificate. If you decide to use a LetsEncrypt Certificate, the DNS-01 challenge is the best method to use, as too many issues can arise from trying to route the HTTP-01 challenge from Cloudfront to your origin.

I know CloudFlare allows self-signed certificates and offers "long lasting" certs (that are not publicly trusted) for origin servers to encrypt the connection into their network. Amazon may have comparable options.

TLDR: You do not need a publicly trusted cert for the path between your server and Cloudfront, and that overly complicates things. You should be able to handle this with a non-public cert. You may want to explore options to have fallback public certs for that server at-the-ready in case you need to jump off Cloudront during an incident, but that should be the only thing you need public certs or Certbot for in this setup.

2 Likes
13

Just a curl request to your domain. See attached txt file. I did not include the whole page it is very long. Note my response headers said "CloudFront: Miss". Maybe your Cloudfront edge for your location has cached the old page?

I largely agree with @jvanasco though. Are you sure you need a cert between the Load Balancer and Apache? I see you have gotten Let's Encrypt certs recently so at least you worked that out.

suhozid.hr.txt (18.9 KB)

2 Likes