Unauthorized Response Type for EC2

Help
#1

My domain is: http://digitalphenotypingmit.com/

Hello. I have been trying to install software known as Radar-base which uses letsencrypt to request the SSL certificate of my domain. However, I'm getting an authorized response type. Here is the output:

==> Requesting Let's Encrypt SSL certificate for digitalphenotypingmit.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for digitalphenotypingmit.com
Using the webroot path /data/letsencrypt for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. digitalphenotypingmit.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://digitalphenotypingmit.com/.well-known/acme-challenge/Cc8Cd4XD_xhPRwOMCp3l7oM2_SG7gInHHh1YV0DZav4 [54.159.14.8]: 503
IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: digitalphenotypingmit.com
   Type:   unauthorized
   Detail: Invalid response from
   http://digitalphenotypingmit.com/.well-known/acme-challenge/Cc8Cd4XD_xhPRwOMCp3l7oM2_SG7gInHHh1YV0DZav4
   [54.159.14.8]: 503

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

Additional info: I'm using an EC2 instance with OS Amazon Linux 2. My domain has an A record that routes traffic to my ELB which is attached to my EC2 instance. My firewall should be accepting all HTTP and HTTPS traffic. Thanks!

#2

Hi @ObitoSigma

there is a http status 503: Why?

If your webserver doesn't work, webroot can't work.

#3

Doesn't look like it's properly attached:

$ curl -X GET -I digitalphenotypingmit.com
HTTP/1.1 503 Service Unavailable: Back-end server is at capacity
Content-Length: 0
Connection: keep-alive

I assume that error is from the ELB.

#4

+1 connection would exceed current capacity threshold (of "0").
LOL

#5

Thank you. Please pardon me for inexperience. Is the 503 error specifically preventing it from working? I thought that my installation would create the webserver.

For testing, I temporarily set up a basic httpd webserver and everything works fine. I did notice that my ELB requires both an HTTP listener (for the ELB DNS to work) and an HTTPS listener (for my domain to work), so if that's a bad thing, maybe that would indicate something's wrong? Thanks again.

1 Like
#6

Is httpd running right now?

According the ELB, it can't connect to your httpd server. That's what the 503 is about.

As such, any further operation (such as trying to access a webpage, or issue a certificate) is going to fail.

3 Likes
#7

No, I set up httpd temporarily just to see if my EC2 instance, ELB, and domain can talk to each other. I tried the installation with httpd running and got the following error:

ERROR: for webserver  Cannot start service webserver: driver failed programming external connectivity on endpoint radar-cp-hadoop-stack_webserver_1 (9df197c39129736ebf32d7879b275af3ae1e1fe5afe63135660078810e1112d4): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use
ERROR: Encountered errors while bringing up the project.

This is what leads me to believe the installation creates the webserver, so I can't have a webserver already running.

#8

OK I think I am starting to understand. Your software's installation process installs the webserver, starts it and then runs Certbot for you. All in one command.

Is it possible that the ELB health check is to blame?

Before you start your software's installation process, your backend is going to be marked as unhealthy, because no webserver is running. Fair enough.

Then you start the installation process, which installs and starts a webserver and then runs Certbot.

However, the ELB health check has not had a chance to pick up on the new webserver. So your backend is still unhealthy, and Let's Encrypt gets a 503.

Is there a way to disable the health check and force your backend to be marked as healthy?

Otherwise, can you run httpd, let the backend become healthy, temporarily set your health check interval to e.g. 1 hour, remove httpd, and then run the installation process?

2 Likes
#9

As much as I love Let's Encrypt, you may find it easier to stay within the AWS ecosystem. You can use their Certificate Manager to get an Amazon-issued certificate for public use on the ELB. If you need HTTPS between your load balance and your back-end EC2 server (which you might need depending on your requirements, but you might want to try starting without to get up and running) it's a bit trickier but they just announced some kind of interesting "Nitro Enclave" technology to let you use their certificates without your server needing the private key using the PKCS11 protocol somehow. I haven't dug into it yet, and maybe it's not applicable for what you're trying to do, but it seems intriguing to me and perhaps an indication of where future "cloud" technology is heading.

3 Likes
#10

@_az That was a really smart suggestion! Unfortunately, it resulted in a 408 error this time.

@petercooperjr I am in possession of an ACM certificate. In the config file for installation, there are two related settings:

ENABLE_HTTPS: Whether the host should host HTTPS. Set this to no if there is a downstream reverse proxy server that arranges HTTPS SSL certificates.Set to yes otherwise.

SELF_SIGNED_CERT: Whether to use a self-signed (unsafe) SSL certificate. If you cannot get a certificate via Letsencrypt, e.g. because you are not reachable from the internet, set this to yes. Otherwise, set this to no.

Are either of these applicable to me if I were to use an ACM certificate? I should be using HTTPS (so "yes"), and an ACM certificate is not self-signed (right?) (so "no"). There doesn't seem to be a config option for me to insert my certificate. I'll ask the dev team behind this application if this is possible.

#11

So, usually the way these kinds of things work (though I'm not extremely familiar with AWS ELB; my comments may have made me seem more knowledgeable than I am; I haven't needed to use ELB for my hobby AWS stuff) is that there are really two connections: one from the user's browser to the ELB, and one from the ELB to the back-end application. If you're going to set up ELB with your ACM certificate, then that covers the connection from the browser to the ELB. In your back-end application, you only need to enable HTTPS if you need to secure the network between the ELB and your back-end server. Probably it's a good idea in general (just because adding the security likely won't hurt anything), but for initially trying to get this up-and-running you may want to try not enabling HTTPS on your back-end application (which is probably turning off that ENABLE_HTTPS setting).

Another option might be using that self-signed cert, and then somehow configuring ELB to trust that certificate when connecting to your server. I don't know if it can do that, though, but it probably can. If so, that's when you'd use the SELF_SIGNED_CERT option, if I understand it correctly.

And then, once the back end is up and accessible to the world via ELB (via one of those two methods), you should be able to have it use Let's Encrypt to issue a publicly-trusted certificate for it, which could then be used to secure the connection from ELB to that back-end server.

Though the best way to set this up may depend on why you're using an ELB at all. If this is going to be having a ton of identically-configured back-end servers, you may run into rate-limit trouble if they each are trying to create their own individual certificates for the same externally-visible hostname (not to mention they won't be able to answer challenges intended for a different back-end).

2 Likes