Ran a packet capture whilst requesting the cert. It gets all the way to the acme challenge from remote servers. My server sends back a 200 OK.
Then I see a BUNCH of SYNs from the same servers that my server never respond to. The remote servers keep retransmitting the SYNs, but after awhile, seeing that I’m not responding, they give up.
I’m very confused. Can’t run certbot manually in the terminal and can’t find LE logs anywhere T_T. Not sure were to go from here
This suggests (to me anyway) you've got quite a "strict" set of firewall rules set up. Let's Encrypt recently has set up Multi-Perspective Validation. This might be a problem for your firewall setup. For Let's Encrypt to validate the challenge properly, it isn't wise to use a "block all, allow some" kind of firewall setup, so I would recommend change it to an "allow all, block some" setup.
Hi, thanks for responding! Normally people take a day or two to respond on forums like this, I totally didn't expect a response this fast.
This suggests (to me anyway) you’ve got quite a “strict” set of firewall rules set up. Let’s Encrypt recently has set up Multi-Perspective Validation. This might be a problem for your firewall setup. For Let’s Encrypt to validate the challenge properly, it isn’t wise to use a “block all, allow some” kind of firewall setup, so I would recommend change it to an “allow all, block some” setup.
It's a VOIP server that's using a responsive Firewall. Unfortunately, due to the nature of the beast, allow all, block some, isn't really an option I'm afraid. So many people try to hammer on it to try to setup their offshore phone accounts (or something, I don't know) that it's not secure that way. The VOIP server is not behind another firewall either, it has its own and that's it.
That said, I could try whitelisting every IP I see in the packet capture... It's DEFINITELY possible that this validation thing is why it's failing, because it has been working for years and it just started failing this month. I'll check it out.
That may be an endless task and unless you are only allowing IPs that try /.well-known/acme-challenge/ requests, your are also allowing the wrong IPs. (not recommended).
In other words, the allowed requests should be URL based NOT IP based.
Is there any way to put this VoIP system behind any firewall?
Sadly no. Yeah, I whitelisted the IPs and it worked. Reading up, people over at FreePBX are saying you just have to open port 80 all the time for automatic, or manually open 80 when we renew every 2 months and closing thereafter. Thanks for the help though guys!