Port visible to test site, but not visible to letsencrypt due to server farm being blocked

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: snafu.ddns.net

I ran this command: sudo letsencrypt certonly --staging

It produced this output:

Domain: snafu.ddns.net
Type: connection
Detail: 65.110.x.x: Fetching
http://snafu.ddns.net/.well-known/acme-challenge/A-pjJl5zQLBVpiq7bQDOXnR1qikWJYFefOgYI7Z6B4s:
Timeout during connect (likely firewall problem)

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.

  • Your account credentials have been saved in your Certbot
    configuration directory at /etc/letsencrypt. You should make a
    secure backup of this folder now. This configuration directory will
    also contain certificates and private keys obtained by Certbot so
    making regular backups of this folder is ideal.

My web server is (include version): Nothing on port 80, used a variety of ways to get this through.

The operating system my web server runs on is (include version): No webserver for outside use.

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 0.31.0

Now that I have that information out of the way, I have a multitude of services running for IoT and various other servers running for more IoT that are not for the "general public".

One of the services actually warns me (rather abruptly, which isn't a bad thing) that some hackbot is attempting to connect to it with an invalid encryption method. When I see that, I get the CIDR of the server farm that it is on and add it to the firewall group that block access to the incoming ports in the router, this means that server farms that allow hackbots to operate wind up being blocked. Well, over the past 2 years, that list is almost 1000 CIDRs, so possibly tens of millions of IP addresses unable to access the incoming ports. The two IP addresses at letsencrypt.org are not in the block list, but, at an outside location (via vpn), I CAN see the port 80 opened by certbot during it's communication (remotely using a port scan during and after it failed), but the letsencrypt validation servers cannot (meaning they reside on a server farm that has been blocked).

This lies the big issue, what server IP addresses are being used for the certificate validation? I need the IP addresses, so I can allow only those from that server farm through to only port 80.

I looked at DNS-01, but the automation of that isn't going to be possible with no-ip domains, that I've seen, if anyone has anything further, I'd be interested to know. But the most important part would be the pool of IP addresses that is used, yes, I know they change, but if that list is kept up to date somewhere, I can update my list on the router when necessary.

2

Hello @GuruSR, welcome to the Let's Encrypt community. :slightly_smiling_face:

Let's Encrypt uses many IP Addresses to validate your domain, likely your firewall is not letting them through. Here are 2 links that I hope will help explain how Let's Encrypt does its work of validating a domain.

And "What IP addresses does Let’s Encrypt use to validate my web server?" is here:

3 Likes
3

OR

You could permit all IPs to port 80, handle the ACME challenge requests there, and forward all other requests to HTTPS [where you can block whatever IPs you like].

3 Likes
4

An easy, secure, method is to use Certbot's pre/post hooks to start/stop firewall rules. Port 80 only needs to be open while the Certificate is being procured, it can remain closed the rest of the time.

3 Likes
5

Is snafu.ddns.net your actual domain name? Because I see an Apache server listening on port 80 right now.

3 Likes
6

Except that...
If port 80 is truly insecure, opening it (for even one minute) is not the best idea.
I'd prefer to secure it in a way that leaving port 80 open wouldn't be a problem.

3 Likes
7

They're not running a webserver. They're most likely running Certbot in standalone mode to obtain certs for IoT devices. The only time they need to open their network to port 80 is when obtaining a Certificate.

2 Likes
9

Then why close the port at all?
That amounts to:
You are not allowed to speak with those that aren't listening.

2 Likes
10

At one point for firewall there was a push of the "belt and suspenders approach"; maybe still lingering.

11

A common setup for the OP's scenario is that port 80 is forwarded to a specific home computer for the express purpose of obtaining LetsEncrypt Certificates. Dropping all port 80 traffic to that computer/port is standard practice, because someone doesn't want to manage/harden that machine as if it were a public internet machine. Dropping bot traffic from entering a network is an ideal too.

For users who do not run webservers, and are concerned with botnet traffic, using Certbot hooks to toggle firewall rules is probably the best and easiest option.

2 Likes
12

Fully understand how it works, the sad part is, hackbots were detected and the entire farm blocked for access to the incoming ports, the problem is, I don't know which IP addresses being used [range] to allow those through. I don't want to lower security to allow the actual validation.

It like all the other incoming ports are lumped into a firewall group, allowing every hackbot to pound that port 80 just for the sake of ACME isn't ideal.

It's a Ubiquiti router that isn't possible, since the firewall rules don't exist on the machine it is on.

Yes, I tossed that up since I figured people would state "the port isn't open", clearly it is, but the letsencrypt validation servers are on a server farm that has been blocked from accessing those same ports, so without me knowing which ones there is, there's 987 CIDR entries to look at to determine which one is causing the blocking. I'm not about to remove those entries.

Oh, I agree, but I left that apache2 open for a bit just so I could make sure people when reading this wouldn't say "Hey, it isn't open, check your firewall settings", but clearly someone saw it. :slight_smile:

It isn't I don't want the port open, I don't want them open to server farms for hackers to play with.

I allow ports to be open, just not to server farms, I mean I'm connecting in from places that are public WiFi, etc, so I can see those ports without issues, right now, 80 is dead again, because I turned the apache2 off and undid the changes to the NAT for port 80 to point it back to the right place. I wonder if certbot itself has a list of the servers it looks for in it somewhere that I could read and get them then.

I think any security that helps provide blocking protection against hacking is a good thing. I can still visit those server farms with website hostings, they just can't visit my location to use ports I don't believe they should be accessing.

13

If you only turn on Apache to get a cert, then you can leave the port always open and leave Apache always turned off.
Furthermore, you could use certbot in --standalone mode and it could serve the challenge request itself; Thus, insuring that no one can reach the potentially insecure web service ("Apache") on port 80 [ever].

3 Likes
14

Maybe consider changing to a different challenge type like DNS-01 challenge Challenge Types - Let's Encrypt

2 Likes
15

@Bruce5051:

You might need to look into other ACME clients.
Like: acme.sh

3 Likes
16

I agree, unless security is so tight that getting certificates cannot happen then weakens security.

2 Likes
17

You seem to be using an old version of Certbot. The current version is v1.30.0 as view from here Tags · certbot/certbot · GitHub

1 Like
18

Those IP addresses are not something that is available for Let's Encrypt validation, sorry.
Maybe time to look to another Certificate Authority for certificates.

What IP addresses does Let’s Encrypt use to validate my web server?

We don’t publish a list of IP addresses we use to validate, and these IP addresses may change at any time. Note that we now validate from multiple IP addresses.

That is taken from here: FAQ - Let's Encrypt

2 Likes
19

It does not, and Certbot doesn’t work like that. LetsEncrypt validates from multiple IPs around the globe, and they are subject to change at any moment.

Certbot and other clients don’t receive any data from LetsEncrypt via port 80, they simply publish a unique challenge on the public internet - which is specified by the ACME server.

I think your only options are:

  • purchase a domain. You can get cheap ones for a few dollars.
  • if your ddns provider supports txt records you can use the dns challenge manually or by delegating it to another provider. Does acme-dns still offer public accounts?

Some people have used workarounds for similar constraints by using CDNs that support rewrite rules. I don’t think any of that stuff is compatible with dynamic dns setups.

2 Likes
20

Might want to look again, bumped to 1.31.0, I did an update, it is current.

Anyhow, I did get the challenge to work on port 80, with a workaround. Thanks for the ideas everyone.

1 Like
21

So you've done that since the initial post show in the above quote?