Verification fails - error 400

My domain is: omega1.b-s-technic.de

I ran this command:
MDaemon Internal Letsencypt Sktript

It produced this output:
Selecting the http-01 challenge and getting challenge data for dns:gamma1.b-s-technic.de.
The challenge status URL is https://acme-v02.api.letsencrypt.org/acme/chall-v3/307619609916/PW_Akw.
The challenge identifier is dns:gamma1.b-s-technic.de.
The URL to verify the challenge is gamma1.b-s-technic.de/.well-known/acme-challenge/l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao.
The Challenge file name for dns:gamma1.b-s-technic.de is l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao
The Challenge Content for dns:gamma1.b-s-technic.de is l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao.wpLIexSUADQZK-yKEsvDmhdUCuqNjOVl-bnuGw-m17Y
Creating D:\MDaemon\WorldClient\HTML.well-known\Acme-challenge\l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao for dns:gamma1.b-s-technic.de.
Submitting the ACME challenge for dns:gamma1.b-s-technic.de for verification.
Selecting the http-01 challenge and getting challenge data for dns:mst.b-s-technic.de.
The challenge status URL is https://acme-v02.api.letsencrypt.org/acme/chall-v3/307619609926/Qt0U0g.
The challenge identifier is dns:mst.b-s-technic.de.
The URL to verify the challenge is mst.b-s-technic.de/.well-known/acme-challenge/Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E.
The Challenge file name for dns:mst.b-s-technic.de is Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E
The Challenge Content for dns:mst.b-s-technic.de is Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E.wpLIexSUADQZK-yKEsvDmhdUCuqNjOVl-bnuGw-m17Y
Creating D:\MDaemon\WorldClient\HTML.well-known\Acme-challenge\Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E for dns:mst.b-s-technic.de.
Submitting the ACME challenge for dns:mst.b-s-technic.de for verification.
Selecting the http-01 challenge and getting challenge data for dns:omega1.b-s-technic.de.
The challenge status URL is https://acme-v02.api.letsencrypt.org/acme/chall-v3/307619609936/kf9WvQ.
The challenge identifier is dns:omega1.b-s-technic.de.
The URL to verify the challenge is omega1.b-s-technic.de/.well-known/acme-challenge/4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0.
The Challenge file name for dns:omega1.b-s-technic.de is 4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0
The Challenge Content for dns:omega1.b-s-technic.de is 4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0.wpLIexSUADQZK-yKEsvDmhdUCuqNjOVl-bnuGw-m17Y
Creating D:\MDaemon\WorldClient\HTML.well-known\Acme-challenge\4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0 for dns:omega1.b-s-technic.de.
Submitting the ACME challenge for dns:omega1.b-s-technic.de for verification.
Waiting for the order status to update... 0
Error: The challenge did not complete.

            Host Name: gamma1.b-s-technic.de
            Error Code: 400
            Error Type: urn:ietf:params:acme:error:connection
            Error Detail: 217.92.236.233: Fetching http://gamma1.b-s-technic.de/.well-known/acme-challenge/l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao: Connection reset by peer

            Host Name: mst.b-s-technic.de
            Error Code: 400
            Error Type: urn:ietf:params:acme:error:connection
            Error Detail: 217.92.236.233: Fetching http://mst.b-s-technic.de/.well-known/acme-challenge/Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E: Timeout during connect (likely firewall problem)

            Host Name: omega1.b-s-technic.de
            Error Code: 400
            Error Type: urn:ietf:params:acme:error:connection
            Error Detail: 217.92.236.233: Fetching http://omega1.b-s-technic.de/.well-known/acme-challenge/4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0: Timeout during connect (likely firewall problem)

This is a critical error, the script will now stop.
Information obtained from the following URLs: https://acme-v02.api.letsencrypt.org/acme/authz-v3/307619609916 https://acme-v02.api.letsencrypt.org/acme/authz-v3/307619609926 https://acme-v02.api.letsencrypt.org/acme/authz-v3/307619609936

My web server is (include version): MDaemon Internal

The operating system my web server runs on is (include version): Windows 10

My hosting provider, if applicable, is: ISP is Telekom

I can login to a root shell on my machine (yes or no, or I don't know):

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

1 Like

Welcome to the community @fagopon
And thank you for the detailed results

These are slightly different but both are most often caused by a firewall at your facility.

Do you have any firewalls in Windows? Or in your comms gear like routers and similar that might block the IP addresses used by Let's Encrypt?

Have you contacted the MDaemon support to see if something in their software could cause this? Maybe their software is conflicting with other things running on your system so the HTTP request is not getting to it.

3 Likes

The HTTP request works fine. I checked it from another location. Ports are opened. It worked fine for the last months. Now i tried to renewed the certificate. And this happened. Checked the IP is reachable from USA. This was fine too. I contacted the MDaemon reseller also. But actually they have no idea whats wrong.

1 Like

I see your past successful cert history. Something has changed since your last cert issued Oct27

Have you tried rebooting your server? I rarely suggest it but sometimes it helps :slight_smile:

I tested access from US and Germany. One of the Let's Encrypt server farms is there. But, all of that worked fine. I also used Let's Debug test site which uses the Let's Encrypt Staging system as one of its tests and that worked too (link here).

Is there any kind of firewall?. Maybe blocking just some inbound IP addresses? Or, some sort of DDoS firewall that is blocking repeated requests (like from Let's Encrypt servers?).

Do you have ability to see logs of internet traffic into your system? Like at the router if a residential system?

2 Likes

Maybe i updatet Mdaemon since the last Request and or installed Windows Updates.

I checked the http logs of MDaemon.

It looks like Letsencrypt can sucessfull connect to the webserver.

17.58.58.6 - - 24/Jan/2024:16:12:50 +0100 "GET /.well-known/acme-challenge/l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao HTTP/1.1" 200 87
17.58.58.7 - - 24/Jan/2024:16:12:51 +0100 "GET /.well-known/acme-challenge/Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E HTTP/1.1" 200 87
17.58.58.4 - - 24/Jan/2024:16:12:52 +0100 "GET /.well-known/acme-challenge/4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0 HTTP/1.1" 200 87
17.58.57.4 - - 24/Jan/2024:16:13:31 +0100 "GET /.well-known/acme-challenge/l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao HTTP/1.1" 200 87
17.58.57.6 - - 24/Jan/2024:16:13:32 +0100 "GET /.well-known/acme-challenge/Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E HTTP/1.1" 200 87
17.58.57.5 - - 24/Jan/2024:16:13:33 +0100 "GET /.well-known/acme-challenge/4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0 HTTP/1.1" 200 87

Maybe. I would expect to see three sets of challenges for each domain name and those are only two. More importantly, those IP addresses belong to Apple and are not from let's encrypt. Do you have any communications gear from Apple that might be handling requests at your site?

3 Likes

No there is no communication gear from Apple. There is a cisco router RV042 behind a AVM Fritzbox. The Cisco is setup as a exposed host behind the Fritzbox. I will try to check logs in the past how it looked.

Here the HTTP Log from h last succesfull request.

35.90.113.149 - - 27/Oct/2023:09:32:46 +0200 "GET /.well-known/acme-challenge/f59bqXN-4dhqcf84Z1de_CU1VUg8mhMfX107i5BBnfY HTTP/1.1" 200 87
3.14.255.232 - - 27/Oct/2023:09:32:46 +0200 "GET /.well-known/acme-challenge/f59bqXN-4dhqcf84Z1de_CU1VUg8mhMfX107i5BBnfY HTTP/1.1" 200 87
23.178.112.209 - - 27/Oct/2023:09:32:46 +0200 "GET /.well-known/acme-challenge/f59bqXN-4dhqcf84Z1de_CU1VUg8mhMfX107i5BBnfY HTTP/1.1" 200 87
3.14.255.232 - - 27/Oct/2023:09:32:46 +0200 "GET /.well-known/acme-challenge/IcSSsVdQUCRNiQdm7Sd6v07w-1HfwL3kBSF9KztZNtk HTTP/1.1" 200 87
23.178.112.108 - - 27/Oct/2023:09:32:47 +0200 "GET /.well-known/acme-challenge/IcSSsVdQUCRNiQdm7Sd6v07w-1HfwL3kBSF9KztZNtk HTTP/1.1" 200 87
54.149.245.191 - - 27/Oct/2023:09:32:47 +0200 "GET /.well-known/acme-challenge/IcSSsVdQUCRNiQdm7Sd6v07w-1HfwL3kBSF9KztZNtk HTTP/1.1" 200 87
35.90.113.149 - - 27/Oct/2023:09:32:47 +0200 "GET /.well-known/acme-challenge/7MbMRRiGxK_E9fasouMr-5_wSTPkatpmguE6kBf9gZo HTTP/1.1" 200 87
23.178.112.107 - - 27/Oct/2023:09:32:48 +0200 "GET /.well-known/acme-challenge/7MbMRRiGxK_E9fasouMr-5_wSTPkatpmguE6kBf9gZo HTTP/1.1" 200 87
3.145.182.179 - - 27/Oct/2023:09:32:48 +0200 "GET /.well-known/acme-challenge/7MbMRRiGxK_E9fasouMr-5_wSTPkatpmguE6kBf9gZo HTTP/1.1" 200 87

Thats crazy. Maybe its a Routing problem between the locations ?

A routing problem somewhere, yes. The IP addresses in the Oct27 log are normal Let's Encrypt IP. They may change each time but those are LE IP. There are 2 US locations and one Cloudflare magic transit IP.

Your latest log IP look very different. Because of that any routing problem is likely closer to you than nearer each different LE location.

Did you just reset something. Because I got several HTTP connect failures just now but after that can connect again.

What IP shows in the log for these requests?

curl -i omega1.b-s-technic.de/.well-known/acme-challenge/MikeTest1556utc
curl -i gamma1.b-s-technic.de/.well-known/acme-challenge/MikeTest1555utc
3 Likes

System is rebooted now. Heres the last log entries. 3.83.219.221 is the IP from your test.

17.58.58.25 - - 24/Jan/2024:16:43:40 +0100 "GET /.well-known/acme-challenge/l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao HTTP/1.1" 200 87
17.58.58.22 - - 24/Jan/2024:16:43:40 +0100 "GET /.well-known/acme-challenge/Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E HTTP/1.1" 200 87
17.58.58.22 - - 24/Jan/2024:16:43:41 +0100 "GET /.well-known/acme-challenge/4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0 HTTP/1.1" 200 87
17.58.57.102 - - 24/Jan/2024:16:44:10 +0100 "GET /.well-known/acme-challenge/l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao HTTP/1.1" 200 87
17.58.57.101 - - 24/Jan/2024:16:44:11 +0100 "GET /.well-known/acme-challenge/Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E HTTP/1.1" 200 87
17.58.57.104 - - 24/Jan/2024:16:44:12 +0100 "GET /.well-known/acme-challenge/4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0 HTTP/1.1" 200 87
17.58.57.21 - - 24/Jan/2024:16:48:43 +0100 "GET /.well-known/acme-challenge/l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao HTTP/1.1" 200 87
17.58.57.20 - - 24/Jan/2024:16:48:43 +0100 "GET /.well-known/acme-challenge/Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E HTTP/1.1" 200 87
17.58.57.20 - - 24/Jan/2024:16:48:44 +0100 "GET /.well-known/acme-challenge/4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0 HTTP/1.1" 200 87
17.58.58.4 - - 24/Jan/2024:16:51:15 +0100 "GET /.well-known/acme-challenge/l1TdoTcnmNjvzOK0mVjuocHrVa0qiB-QypB4sc8Tpao HTTP/1.1" 200 87
17.58.58.6 - - 24/Jan/2024:16:51:16 +0100 "GET /.well-known/acme-challenge/Iz8xp8HNwMkT0RNup1s6IVzKj65zm_4yLBDSt5NNo2E HTTP/1.1" 200 87
17.58.58.9 - - 24/Jan/2024:16:51:16 +0100 "GET /.well-known/acme-challenge/4uS-VyYr2bQ5Ftrv5qKGJUs3UzMCtrXAOiRUyGoL8H0 HTTP/1.1" 200 87
23.95.63.196 - - 24/Jan/2024:16:52:28 +0100 "GET /.env HTTP/1.1" 404 93
23.95.63.196 - - 24/Jan/2024:16:52:29 +0100 "POST / HTTP/1.1" 200 2530
3.83.219.221 - - 24/Jan/2024:16:56:54 +0100 "GET /.well-known/acme-challenge/MikeTest1556utc HTTP/1.1" 404 93
3.83.219.221 - - 24/Jan/2024:16:57:01 +0100 "GET /.well-known/acme-challenge/MikeTest1556utc HTTP/1.1" 404 93
3.83.219.221 - - 24/Jan/2024:16:57:04 +0100 "GET /.well-known/acme-challenge/MikeTest1555utc HTTP/1.1" 404 93

None of those IPs are from LE:

They are all from Apple. See: ARIN Whois/RDAP - American Registry for Internet Numbers
Apple basically crawls the web [because they can].

If you want to get a better understanding of who/what is making those requests, you should add the UserAgent to the logs.

2 Likes

That is the IP I used for that test. It was from an AWS EC2 server on the US East Coast (I am no longer using that IP). So, at least whatever this Apple thing is wasn't involved with that.

Yeah, those latest IP from Apple are very similar to the IP you showed in your 16:12 log

Agree with @rg305 would be interested to see the user-agent in the log record too.

2 Likes

Sure, but why would they use perfectly formed ACME Challenge URI? And for what looks like a repeating pattern for 3 domain names?

2 Likes

Because they must have been posted somewhere.
Like in the topic's first post - LOL

[thus the crawling]

Let's see how fast this gets crawled:
http://mst.b-s-technic.de/.well-known/acme-challenge/nosycrawlercrawling

2 Likes

Good point !

Can watch the logs for anyone crawling that too. Which is why I added the utc time in the URI :slight_smile:

1 Like

Problem resolved. ESET Endpoint Antivirus has a Network Filter which blocked the Letsencrypt Traffic.

Thank you so much for the help.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.