404 Error on AWS Elastic Beanstalk after Installing a Certificate

jdcoulte · June 22, 2021, 3:22am

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: https://app.plprx.com/

I ran this command: N/a, I was able to successfully install a certificate (after a number of attempts)

It produced this output: I am now getting an "Error 521 Web Server is Down" response when trying to visit app.plprx.com and I am getting a 404 not found when I try to use the original AWS Elastic Beanstalk URL: http://plp-test.eba-r4ct3cgf.us-west-2.elasticbeanstalk.com/

My web server is (include version): nginx/1.20.0

The operating system my web server runs on is (include version): AWS Linux 2 AMI

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 1.16.0

I'm new to installing SSL certificates and followed the instructions on the certbot site. After several attempts I was able to get the certificate to install, but ever since I'm getting the server down when visiting app.plprx.com and I'm getting a 404 not found when I use the AWS Elastic Beanstalk URL. My site is hosted on GoDaddy but runs through Cloudflare, and then I have a CNAME DNS entry to the elastic beanstalk URL.

This is the output in my server log files: Not Found: /.well-known/acme-challenge/Am8X8KUO1xJzWiIjKjaKq_APsxYPQO07-gADgsz5-5c

When I run certbot certificates this is the response:

Found the following certs:
Certificate Name: app.plprx.com
Serial Number: xxxx
Key Type: RSA
Domains: app.plprx.com
Expiry Date: 2021-09-20 01:44:05+00:00 (VALID: 89 days)
Certificate Path: /etc/letsencrypt/live/app.plprx.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/app.plprx.com/privkey.pem

griffin · June 22, 2021, 4:00am

Welcome to the Let's Encrypt Community, Jeff

What is the output of sudo nginx -T on your webserver?

jdcoulte · June 22, 2021, 4:05am

Hi Griffin, thanks for the kind welcome

I'm getting this:
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

I suspect the issue may be somewhere in the http or server settings because that's what I had to fiddle around with to get the certificate to install in the first place. This is that section from the nginx -T command:

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    include       conf.d/*.conf;

    map $http_upgrade $connection_upgrade {
        default     "upgrade";
    }

    server {
        access_log    /var/log/nginx/access.log main;

        client_header_timeout 60;
        client_body_timeout   60;
        keepalive_timeout     60;
        gzip                  off;
        gzip_comp_level       4;
        gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;

server_name app.plprx.com plp-test.eba-r4ct3cgf.us-west-2.elasticbeanstalk.com;

# Include the Elastic Beanstalk generated locations
        include conf.d/elasticbeanstalk/*.conf;
    
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/app.plprx.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/app.plprx.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = app.plprx.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


        listen        80 default_server;

server_name app.plprx.com plp-test.eba-r4ct3cgf.us-west-2.elasticbeanstalk.com;
    return 404; # managed by Certbot


}}

jdcoulte · June 22, 2021, 4:06am

Not sure if that pasted correctly so let me know if it doesn't make sense. I believe I added the http://plp-test.eba-r4ct3cgf.us-west-2.elasticbeanstalk.com/ server_name entry AFTER I was already having issues and that did not resolve anything.

griffin · June 22, 2021, 4:06am

Please, if you can, post the entire output with 3 backticks above and below, like this:

```
output
```

I know it's very long. It just prevents missing little things.

rg305 · June 22, 2021, 4:07am

done

griffin · June 22, 2021, 4:08am

Thanks, @rg305.

This additional waste of a sentence added to exceed 20 characters.

griffin · June 22, 2021, 4:09am

@jdcoulte

You can also just sudo nginx -T > output.txt then upload the output.txt file. There's usually nothing sensitive in there.

jdcoulte · June 22, 2021, 4:24am

I'm a new user so it won't let me attach but here's the full output:

# configuration file /etc/nginx/nginx.conf:
#Elastic Beanstalk Nginx Configuration File

user                    nginx;
error_log               /var/log/nginx/error.log warn;
pid                     /var/run/nginx.pid;
worker_processes        auto;
worker_rlimit_nofile    32633;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    include       conf.d/*.conf;

    map $http_upgrade $connection_upgrade {
        default     "upgrade";
    }

    server {
        access_log    /var/log/nginx/access.log main;

        client_header_timeout 60;
        client_body_timeout   60;
        keepalive_timeout     60;
        gzip                  off;
        gzip_comp_level       4;
        gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;

	server_name app.plprx.com plp-test.eba-r4ct3cgf.us-west-2.elasticbeanstalk.com;

	# Include the Elastic Beanstalk generated locations
        include conf.d/elasticbeanstalk/*.conf;
    
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/app.plprx.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/app.plprx.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}


    server {
    if ($host = app.plprx.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


        listen        80 default_server;

	server_name app.plprx.com plp-test.eba-r4ct3cgf.us-west-2.elasticbeanstalk.com;
    return 404; # managed by Certbot


}}

# configuration file /etc/nginx/mime.types:

types {
    text/html                                        html htm shtml;
    text/css                                         css;
    text/xml                                         xml;
    image/gif                                        gif;
    image/jpeg                                       jpeg jpg;
    application/javascript                           js;
    application/atom+xml                             atom;
    application/rss+xml                              rss;

    text/mathml                                      mml;
    text/plain                                       txt;
    text/vnd.sun.j2me.app-descriptor                 jad;
    text/vnd.wap.wml                                 wml;
    text/x-component                                 htc;

    image/png                                        png;
    image/svg+xml                                    svg svgz;
    image/tiff                                       tif tiff;
    image/vnd.wap.wbmp                               wbmp;
    image/webp                                       webp;
    image/x-icon                                     ico;
    image/x-jng                                      jng;
    image/x-ms-bmp                                   bmp;

    font/woff                                        woff;
    font/woff2                                       woff2;

    application/java-archive                         jar war ear;
    application/json                                 json;
    application/mac-binhex40                         hqx;
    application/msword                               doc;
    application/pdf                                  pdf;
    application/postscript                           ps eps ai;
    application/rtf                                  rtf;
    application/vnd.apple.mpegurl                    m3u8;
    application/vnd.google-earth.kml+xml             kml;
    application/vnd.google-earth.kmz                 kmz;
    application/vnd.ms-excel                         xls;
    application/vnd.ms-fontobject                    eot;
    application/vnd.ms-powerpoint                    ppt;
    application/vnd.oasis.opendocument.graphics      odg;
    application/vnd.oasis.opendocument.presentation  odp;
    application/vnd.oasis.opendocument.spreadsheet   ods;
    application/vnd.oasis.opendocument.text          odt;
    application/vnd.openxmlformats-officedocument.presentationml.presentation
                                                     pptx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
                                                     xlsx;
    application/vnd.openxmlformats-officedocument.wordprocessingml.document
                                                     docx;
    application/vnd.wap.wmlc                         wmlc;
    application/x-7z-compressed                      7z;
    application/x-cocoa                              cco;
    application/x-java-archive-diff                  jardiff;
    application/x-java-jnlp-file                     jnlp;
    application/x-makeself                           run;
    application/x-perl                               pl pm;
    application/x-pilot                              prc pdb;
    application/x-rar-compressed                     rar;
    application/x-redhat-package-manager             rpm;
    application/x-sea                                sea;
    application/x-shockwave-flash                    swf;
    application/x-stuffit                            sit;
    application/x-tcl                                tcl tk;
    application/x-x509-ca-cert                       der pem crt;
    application/x-xpinstall                          xpi;
    application/xhtml+xml                            xhtml;
    application/xspf+xml                             xspf;
    application/zip                                  zip;

# configuration file /etc/nginx/conf.d/healthd_logformat.conf:
log_format healthd  '$msec"$uri"'
                    '$status"$request_time"$upstream_response_time"'
                    '$http_x_forwarded_for';
# configuration file /etc/nginx/conf.d/elasticbeanstalk/00_application.conf:
location / {
    proxy_pass          http://127.0.0.1:8000;
    proxy_http_version  1.1;

    proxy_set_header    Connection          $connection_upgrade;
    proxy_set_header    Upgrade             $http_upgrade;
    proxy_set_header    Host                $host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
}

# configuration file /etc/nginx/conf.d/elasticbeanstalk/01_static.conf:
location /static {
    alias /var/app/current/static;
    access_log off;
}

# configuration file /etc/nginx/conf.d/elasticbeanstalk/healthd.conf:
if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
    set $year $1;
    set $month $2;
    set $day $3;
    set $hour $4;
}

access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour healthd;
# configuration file /etc/letsencrypt/options-ssl-nginx.conf:
# This file contains important security parameters. If you modify this file
# manually, Certbot will be unable to automatically provide future security
# updates. Instead, Certbot will print and log an error message with a path to
# the up-to-date file that you will need to refer to when manually updating
# this file.

ssl_session_cache shared:le_nginx_SSL:10m;
ssl_session_timeout 1440m;
ssl_session_tickets off;

ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers off;

ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHAC
HA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";

rg305 · June 22, 2021, 8:20am

What is the purpose of this section/file?:

jdcoulte:

# configuration file /etc/nginx/conf.d/elasticbeanstalk/00_application.conf:
location / {
    proxy_pass          http://127.0.0.1:8000;
    proxy_http_version  1.1;

    proxy_set_header    Connection          $connection_upgrade;
    proxy_set_header    Upgrade             $http_upgrade;
    proxy_set_header    Host                $host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
}

jdcoulte · June 22, 2021, 8:22am

I honestly do not know, it appears that was added automatically by elastic beanstalk (AWS) as part of configuration.

jdcoulte · June 22, 2021, 8:28am

Based on a quick Google I did come across this forum: PHP on Amazon Linux 2 platform · Issue #13 · aws/elastic-beanstalk-roadmap · GitHub

It is a different topic but there is one user who had these exact same "location" settings for their django app, but later in the thread "proxy_pass" is gone and replaced with

proxy_cache_bypass   $http_upgrade;

I'll give that a try and report back.

jdcoulte · June 22, 2021, 8:39am

Unfortunately same result.

griffin · June 22, 2021, 5:15pm

I'll look into this today.

There are certainly more than a few issues.

rg305 · June 22, 2021, 6:26pm

~~It isn't found within any server block.~~

jdcoulte · June 22, 2021, 9:47pm

Isn't it pulled in by this line from the first server block?

# Include the Elastic Beanstalk generated locations
include conf.d/elasticbeanstalk/*.conf;

jdcoulte · June 22, 2021, 9:48pm

Any help would be greatly appreciated. This is my first production application and I'm completely lost

griffin · June 22, 2021, 9:59pm

Dealing with several things right now. Will help as soon as I'm free.

jdcoulte · June 22, 2021, 10:39pm

No rush, this project is still in the testing phase and I've got plenty of other stuff to work on

system · July 22, 2021, 10:39pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.