If what you're running within the EC2 is nginx (or perhaps some other software that supports PKCS11 though I don't think much does), then you may want to take a look at AWS Certificate Manager for Nitro Enclaves, which doesn't let you access the private key directly but has an interface for a program running on the EC2 instance to utilize it.
I was just now retrying the terraform call from within a machine in my AWS account. I tried it 3 times and turns out the error messages i received were different to the ones I got locally:
edit: Which indicates some connection problem. Maybe that was the problem all along? I'll make another test tomorrow with a different network setup.
Exactly. I wasn't gonna regenerate the certificate every time, but store it somewhere safely (e.g. AWS Secret Manager) and just have the EC2 use it.
Another option, or at least what I've done, is to run a Lambda to do the renewal (via DNS-01 challenges) and store it securely (I just use S3, though Secrets Manager would probably be wiser), entirely separately from the systems using it. Then the systems using it just need to get the certificate from the secure storage each time. I've got the system I hacked together posted at https://cooperjr.name/2020/08/13/acme-lambda-renewal/, though be aware that it's more of a proof-of-concept you could use as a starting point for your own development than an off-the-shelf client designed for the public.
I don't know. My thought was Terraform was just reporting the error message it got from the Let's Encrypt server. If that was true the connection is between the LE server and Route53 so not affected by your requester location.
But, if @petercooperjr's post #8 is correct then Terraform could be doing its own validation of the path to Route53 and failing for the various reasons shown. If this is true the requester network setup matters. This would also explain why we can see your TXT record from other locations and no one else has reproduced the error message.
Mind you, I am just theorizing. Is it possible to ask Terraform if they do this?
I still think a load balancer is viable. You are in part reinventing a wheel. It might be a better solution - not just a work-around.
No, but it does limit your authorization choices to just DNS challenges.
Isn't that why the connectivity is the problem? The ACME client runs locally. So there might be some weird vm dns settings that do not allow the connection.
Anyways, I tried the entire process from a machine within the same AWS account, allowing all traffic in the security groups (for now). Turns out it works.
Happy that we solved the issue, even though as you suggested @MikeMcQ I'll sick to the loadbalancer setup. Thank you very much for your support everyone. Nice community.
Any last thoughts?
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.