Expired Let's Encrypt certs on Icewarp - AGAIN

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: mail.topperfloats.com, go2email.topperfloats.com

I ran this command:
certbot --standalone

It produced this output:
C:\Program Files (x86)\Certbot\bin>certbot certonly --standalone
Saving debug log to C:\Certbot\log\letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Please enter in your domain name(s) (comma and/or space separated) (Enter 'c'
to cancel): mail.topperfloats.com, go2email.topperfloats.com
Requesting a certificate for mail.topperfloats.com and go2email.topperfloats.com
Performing the following challenges:
http-01 challenge for go2email.topperfloats.com
http-01 challenge for mail.topperfloats.com
Waiting for verification...
e[31mChallenge failed for domain go2email.topperfloats.come[0m
e[31mChallenge failed for domain mail.topperfloats.come[0m
http-01 challenge for go2email.topperfloats.com
http-01 challenge for mail.topperfloats.com
Cleaning up challenges
e[31mSome challenges have failed.e[0m
e[1m
IMPORTANT NOTES:
e[0m - The following errors were reported by the server:

Domain: go2email.topperfloats.com
Type: unauthorized
Detail: Invalid response from
http://go2email.topperfloats.com/.well-known/acme-challenge/LmT95-gIV1WtllhTA_WmEuzNUtT4ajQVdOCrbWjTbPc
[74.85.224.14]: "404 Not
Found<table width="400" cellpadding="3"
cellspacing="5"><td align="left" val"

Domain: mail.topperfloats.com
Type: unauthorized
Detail: Invalid response from
http://mail.topperfloats.com/.well-known/acme-challenge/yOkymltQJMzm9X4F385giO9O7vvx8obspX7J2XUK85A
[74.85.224.14]: "404 Not
Found<table width="400" cellpadding="3"
cellspacing="5"><td align="left" val"

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

My web server is (include version):
no webserver
C:\Program Files (x86)\Certbot\bin>netstat -ano | findstr ":80"
TCP 0.0.0.0:80 0.0.0.0:0 LISTENING 7044
TCP 74.85.224.14:80 64.62.250.100:36888 TIME_WAIT 0
TCP 74.85.224.14:80 64.62.250.100:38296 TIME_WAIT 0

The operating system my web server runs on is (include version):
MS Server 2016 Essentials Version 1607 (build 14393.2273)

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know):
YES

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
NO - using RDP

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 1.13.0
cd\

I have created IPV6 DNS AAAA records pointing to my domains. At letsdebug.net, the DNS-01 test returns:
All OK!

OK

No issues were found with mail.topperfloats.com. If you are having problems with creating an SSL certificate, please visit the Let's Encrypt Community forums and post a question there.

HTTP-01 test gives me:
AAAANotWorking

ERROR

mail.topperfloats.com has an AAAA (IPv6) record (2607:f650:0:1::10) but a test request to this address over port 80 did not succeed. Your web server must have at least one working IPv4 or IPv6 address. You should either ensure that validation requests to this domain succeed over IPv6, or remove its AAAA record.

A timeout was experienced while communicating with mail.topperfloats.com/2607:f650:0:1::10: Get "http://mail.topperfloats.com/.well-known/acme-challenge/letsdebug-test": context deadline exceeded

Trace:
@0ms: Making a request to http://mail.topperfloats.com/.well-known/acme-challenge/letsdebug-test (using initial IP 2607:f650:0:1::10)
@0ms: Dialing 2607:f650:0:1::10
@10000ms: Experienced error: context deadline exceeded

Ping -6 mail.topperfloats.com returns:
C:\Program Files (x86)\Certbot\bin>ping -6 mail.topperfloats.com

Pinging mail.topperfloats.com [2607:f650:0:1::10] with 32 bytes of data:
Reply from 2607:f650:0:1::10: time<1ms
Reply from 2607:f650:0:1::10: time<1ms
Reply from 2607:f650:0:1::10: time<1ms
Reply from 2607:f650:0:1::10: time<1ms

CONTINUE HERE---------------------
The http-01 trace keeps looking like its trying to validate a web server by looking for a document in the web folders (.well-known......) The DNS-01 trace works so what am I doing wrong?

Thanks, Tom

1 Like
2

You're not using the dns-01 challenge, so using Let's Debug to check that challenge is rather futile.

Pinging the IPv6 address from the host itself or the same network is not a guarantee it's actually working from the world wide web.

Also, I'm surprised the error actually contains a HTTP answer, albeit being the incorrect one: I can't connect to your host at all, not through IPv4 or IPv6.

Also, the fact you're using the standalone authenticator in combination with a <table width="400" cellpadding="3" cellspacing="5"><td align="left" val" contents of the 404 File Not Found-error tells me the authorization request is ending up at a different HTTP server than the internal webserver of certbots standalone plugin, because the standalone plugin either returns the correct token or doesn't output anything at all. It doesn't send 404 file not found errors. Are you running certbot on the actual host of your website? Is there still a webserver running? On which port is the standalone plugin listening? 80 or differently?

1 Like
3

There is no web server running. The output I posted came directly from certbot.

4

The error in the certbot output after Detail: Invalid response from ... is from the Let's Encrypt validation server. It sees HTML (the <table> stuff) as content of a "404 Not Found" error: that error and <table> is NOT send by the standalone server (I just tested that, the plugin, when queried for a challenge which is not existing, says:

:1 - - Incoming request
::1 - - No resources to serve
::1 - - /.well-known/acme-challenge/1234 does not correspond to any resource. ignoring

with cURL output:

server ~ # curl -v -H "Host: example.com" http://localhost:8080/.well-known/acme-challenge/1234
*   Trying ::1:8080...
* Connected to localhost (::1) port 8080 (#0)
> GET /.well-known/acme-challenge/1234 HTTP/1.1
> Host: example.com
> User-Agent: curl/7.74.0
> Accept: */*
>
* Empty reply from server
* Connection #0 to host localhost left intact
curl: (52) Empty reply from server
server ~ #

(I made certbot listening on port 8080 so I didn't have to stop my own webservers, but it's of course valid for testing the standalone plugin output.)

As you can see: no <table>, no 404 Not Found error.

So SOMETHING must be producing that 404 Not Found with <table> HTML contents error if it isn't certbots standalone plugin.

Also:

If there is no webserver listening, why is there something listening on port 80?

2 Likes
5

I'm sorry bug I'm in WAY over my head now....

I hadn't stopped the IW control module. Now netstat -ano | find ":80" shows no listeners.

Does that help?

1 Like
6

It should. You could try again. However, certbot should probably have errored out mentioning it couldn't bind to port 80. But try again and see what happens.

1 Like
7

Failed again "fetching"

Says it's looking for a file:

Domain: mail.topperfloats.com
Type: connection
Detail: Fetching
http://mail.topperfloats.com/.well-known/acme-challenge/VaenwclZXZX4dGqL-7wKbSyVe7HpgfEdat6nLLk224w:
Timeout during connect (likely firewall problem)

How is it trying to find the file if there is no web server?

8

At one point I got and error about my IPV6 address was not publicly routable (we don't use IPV6 generally). So host assigned me this one:

2607:f650:0:1::10

DNS Made Easy validated AAAA record.

My A records point to 74.85.224.14

1 Like
9

Ideally the certificate authority will be able to connect to

That is, Certbot (when run with --standalone) is creating a temporary web server of its own (on port 80 of the computer where you run Certbot) in order to receive the incoming HTTP connections that the certificate authority makes to try to check that the person running Certbot really controls the domain names in the certificate.

https://certbot.eff.org/docs/using.html#getting-certificates-and-choosing-plugins

1 Like
10

What is that IPv6 address pointing to? Is it the same computer that you're running certbot --standalone on?

1 Like
11

IPV6 according to my AAAA records point to:
mail.topperfloats.com
go2email.topperfloats.com

FWIW - certbot installed in c:\certbot but in order to run it, I have to go to c:\program files (x86)\certbot\bin

Could that be part of the problem?

12

I even have rules in my firewall specifically allowing TCP/UDP traffic on ports 80, 443.

13

Does your IceWarp actually receive any mail at all? I can't connect to SMTP port 25, SMTPS port 465, SMTP port 587, POP3 port 110, POP3S port 995, IMAP port 143 or IMAPS port 993 also. Only timeouts. Your whole host is down.

1 Like
14

Thunderbird just sent/received message from Icewarp via IMAP on port 993

I've shutdown those Icewarp services using port 80 (webmail, antispam, antivirus)

15

Usually, I'll get a "Connection refused" if there isn't listening any service on a certain port. Are you SURE your firewall isn't blocking anything else than IceWarp?

1 Like
16

IPV6 according to my AAAA records point to:
mail.topperfloats.com
go2email.topperfloats.com

What devices have the IP addresses 74.85.224.14 and 2607:f650:0:1::10?
Are these addresses used by the same machine where you're running
Certbot?

FWIW - certbot installed in c:\certbot but in order to run it, I have to go to c:\program files (x86)\certbot\bin

Could that be part of the problem?

Nope, that shouldn't matter!

1 Like
17

I guess I could turn it off for a moment. Looking at my FW logs I see that my server is trying to do DNS lookups (port 53) which, long ago, I had blocked outbound DNS because of the reflective DNS hacks. For some reason, it's still blocking outbound port 53 which, I suppose, is only a problem when the server needs to use it. I've even created outbound rules to allow port 53 but it's still being blocked.

Just discovered I had a block on incoming port 53 but I'm not using this box as DNS server. Will try certbot again.

1 Like
18

Same issue. Here's latest log:

C:\Program Files (x86)\Certbot\bin>certbot certonly --standalone
Saving debug log to C:\Certbot\log\letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Please enter in your domain name(s) (comma and/or space separated) (Enter 'c'
to cancel): mail.topperfloats.com, go2email.topperfloats.com
Requesting a certificate for mail.topperfloats.com and go2email.topperfloats.com
Performing the following challenges:
http-01 challenge for go2email.topperfloats.com
http-01 challenge for mail.topperfloats.com
Waiting for verification...
e[31mChallenge failed for domain go2email.topperfloats.come[0m
e[31mChallenge failed for domain mail.topperfloats.come[0m
http-01 challenge for go2email.topperfloats.com
http-01 challenge for mail.topperfloats.com
Cleaning up challenges
e[31mSome challenges have failed.e[0m
e[1m
IMPORTANT NOTES:
e[0m - The following errors were reported by the server:

Domain: go2email.topperfloats.com
Type: connection
Detail: Fetching
http://go2email.topperfloats.com/.well-known/acme-challenge/lHLHV7cW-qfC6LtnyRNsR9IHMf_aL0NSqjYR1z1TOGk:
Timeout during connect (likely firewall problem)

Domain: mail.topperfloats.com
Type: connection
Detail: Fetching
http://mail.topperfloats.com/.well-known/acme-challenge/BA4Uba5X0RzyJy4kfIVhSegjo7BQUXbZcypdggdQmfk:
Timeout during connect (likely firewall problem)

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.

1 Like
19

IPV6 pinging my domain from my local PC

Pinging mail.topperfloats.com [2607:f650
Reply from 2607:f650:0:1::10: time=24ms
Reply from 2607:f650:0:1::10: time=26ms
Reply from 2607:f650:0:1::10: time=18ms
Reply from 2607:f650:0:1::10: time=20ms

20

Perhaps a regional firewall of sorts?