HTTP challenge failing Nginx HTTP to HTTPS redirect

My domain is: delta-tech.com

I ran this command: certbot renew -v --dry-run --debug-challenges

It produced this output:

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: imap.delta-tech.com
  Type:   connection
  Detail: During secondary validation: 216.82.57.6: Fetching http://imap.delta-tech.com/.well-known/acme-challenge/9xCmTWR9VnHGQ4dtSnFHLd4K1UY5FSS93t3ZmkRH_Yg: Timeout during connect (likely firewall problem)

  Domain: mail01.delta-tech.com
  Type:   connection
  Detail: During secondary validation: 216.82.57.6: Fetching http://mail01.delta-tech.com/.well-known/acme-challenge/XNzEMZsJjpWnckA_YrWrI42VY3nV73oOaxdF5AhPWJE: Timeout during connect (likely firewall problem)

  Domain: smtp.delta-tech.com
  Type:   connection
  Detail: During secondary validation: 216.82.57.6: Fetching http://smtp.delta-tech.com/.well-known/acme-challenge/ql9ZCHHCbmdlvuPxjhfgC9JkNn7hMA_9Gy9Zv5fIR8M: Timeout during connect (likely firewall problem)

  Domain: webmail.delta-tech.com
  Type:   connection
  Detail: During secondary validation: 216.82.57.6: Fetching http://webmail.delta-tech.com/.well-known/acme-challenge/vvMbzJZ8Vsqx7vFTOCHFQnmCKMp19jOxUN4rEI7kJrM: Timeout during connect (likely firewall problem)

My web server is (include version): Nginx 1.18.0

The operating system my web server runs on is (include version): Ubuntu Server 22.04.5

My hosting provider, if applicable, is: Self Hosted

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.21.0

I have a Letsencrypt certificate that is securing my email and webmail services. I have setup a cron script to automatically renew the certificate via HTTP (HTTP is redirected to HTTPS). Unfortunately, the renewal is failing secondary validation. I have done the following to troubleshoot the problem:

  1. Verified that the http://mydomain/.well-known/acme-challenge is accessible
$ echo 'test' > /var/www/html/.well-known/acme-challenge/test.html

$ wget -v http://webmail.delta-tech.com/.well-known/acme-challenge/test.html
--2024-10-03 09:49:05--  http://webmail.delta-tech.com/.well-known/acme-challenge/test.html
Resolving webmail.delta-tech.com (webmail.delta-tech.com)... 192.168.100.4
Connecting to webmail.delta-tech.com (webmail.delta-tech.com)|192.168.100.4|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://webmail.delta-tech.com/.well-known/acme-challenge/test.html [following]
--2024-10-03 09:49:05--  https://webmail.delta-tech.com/.well-known/acme-challenge/test.html
Connecting to webmail.delta-tech.com (webmail.delta-tech.com)|192.168.100.4|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5 [text/html]
Saving to: β€˜test.html.1’

test.html.1                                     100%[=====================================================================================================>]       5  --.-KB/s    in 0s      

2024-10-03 09:49:05 (3.40 MB/s) - β€˜test.html.1’ saved [5/5]

  1. Verified that certbot is creating challenge files in the correct directory by running the following in two different terminals:
Terminal 1:
certbot renew --dry-run --debug

When the above command is run, I can see the challenge files being created in the correct directory.

Terminal 2:
very 0.5: ls -al /var/www/html/.well-known/acme-challenge                                                                                                    mta01: Thu Oct  3 10:22:17 2024

total 32
-rw-r--r-- 1 root     root     87 Oct  3 10:22 -95p7GpqkfO2Vl8PsddHLhzPhrDy8Y9W4QSDXSsJSkg
drwxr-xr-x 2 www-data www-data  7 Oct  3 10:22 .
drwxr-xr-x 3 www-data www-data  3 Sep 30 15:42 ..
-rw-r--r-- 1 root     root     87 Oct  3 10:22 RxnzuHMeRs0gBd0J15E3__v3-6bNuzzbZv1Evu7ggz0
-rw-r--r-- 1 root     root     87 Oct  3 10:22 mqc2s_lbgPTEun7CnQ3hOxxAdBExRzRkbYGIvW4bKp4
-rw-r--r-- 1 root     root      5 Oct  2 17:50 test.html
-rw-r--r-- 1 root     root     87 Oct  3 10:22 uW17sSv2ATLHoqFW-nxIGAwmQmKAkImtnbZud6w_N0U

At this point, I am not sure what to troubleshoot next. After all the challenge files are being created in the correct directory and are accessible via HTTP (with HTTPS redirection) at the correct URL. I have also verified that it is not a file permission problem. Any help fixing this problem would be greatly appreciated.

and

suggests the LE ACME validation server cannot reach your server from the remote vantage points used by the multiple-vantage point verification using just HTTP. It doesn't even reach the stage of HTTP to HTTPS redirect.

Are you by any chance geoblocking any country/region?

3 Likes

Definitely looks like some sort of geographic block, where the US reports a 503, and other places can't connect at all:

You may find this FAQ useful:

5 Likes

I am using GeoIP blocking, but I don't think that is causing the problem.

I did a packet capture on the firewall. I can see where Letsencrypt connects to port 80 and gets a 301 redirect. After attempting each domain renewal via HTTP, it then switches to HTTPS. I am not very familiar with troubleshooting TLS v1.3, but it looks like the connection proceeds normally. I am pretty sure that this is the case since the redirect works fine when using a browser.

Packet Capture HTTP

tshark -r letsencrypt_capture.pcap  -Y "(ip.src_host == outbound1.letsencrypt.org || ip.dst_host == outbound1.letsencrypt.org) && tcp.port == 80"

3472  19.602232 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 48123 β†’ 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=3694160449 TSecr=0 WS=128
 3473  19.602304 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 80 β†’ 48123 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=3573006503 TSecr=3694160449
 3479  19.655643 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48123 β†’ 80 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=3694160502 TSecr=3573006503
 3480  19.656078 outbound1.letsencrypt.org β†’ vpn.delta-tech.com HTTP 337 GET /.well-known/acme-challenge/Z2cNYL4TmB_9JzjUB9nmXuOVhEH5XAM66lpp8gf4s7k HTTP/1.1 
 3481  19.656094 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 48123 [ACK] Seq=1 Ack=272 Win=65536 Len=0 TSval=3573006557 TSecr=3694160502
 3482  19.657342 vpn.delta-tech.com β†’ outbound1.letsencrypt.org HTTP 766 HTTP/1.1 301 Moved Permanently  (text/html)

Packet Capture HTTPS

tshark -r letsencrypt_capture.pcap  -Y "(ip.src_host == outbound1.letsencrypt.org || ip.dst_host == outbound1.letsencrypt.org) && tcp.port == 443"

3501  19.760358 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 61327 β†’ 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=3694160607 TSecr=0 WS=128
 3502  19.760406 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 443 β†’ 61327 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=3459792963 TSecr=3694160607
 3509  19.814353 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61327 β†’ 443 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=3694160661 TSecr=3459792963
 3513  19.815066 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1 327 Client Hello (SNI=imap.delta-tech.com)
 3514  19.815087 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61327 [ACK] Seq=1 Ack=262 Win=65536 Len=0 TSval=3459793018 TSecr=3694160661
 3517  19.821509 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 1514 Server Hello, Change Cipher Spec, Application Data
 3518  19.821542 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 1514 Application Data
 3519  19.821553 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 366 Application Data, Application Data
 3523  19.844201 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 61337 β†’ 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=3694160691 TSecr=0 WS=128
 3524  19.844289 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 443 β†’ 61337 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=723317719 TSecr=3694160691
 3531  19.875541 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61327 β†’ 443 [ACK] Seq=262 Ack=1449 Win=64128 Len=0 TSval=3694160722 TSecr=3459793024
 3532  19.875555 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61327 β†’ 443 [ACK] Seq=262 Ack=2897 Win=63616 Len=0 TSval=3694160722 TSecr=3459793024
 3533  19.875563 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61327 β†’ 443 [ACK] Seq=262 Ack=3197 Win=63360 Len=0 TSval=3694160722 TSecr=3459793024
 3534  19.876228 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 146 Change Cipher Spec, Application Data
 3535  19.876244 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61327 [ACK] Seq=3197 Ack=342 Win=65792 Len=0 TSval=3459793079 TSecr=3694160723
 3536  19.876254 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 467 Application Data
 3537  19.876261 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61327 [ACK] Seq=3197 Ack=743 Win=65408 Len=0 TSval=3459793079 TSecr=3694160723
 3538  19.877036 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 353 Application Data
 3539  19.877215 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 353 Application Data
 3540  19.877534 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 677 Application Data
 3541  19.877658 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 90 Application Data
 3542  19.877798 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61327 [FIN, ACK] Seq=4406 Ack=743 Win=65792 Len=0 TSval=3459793080 TSecr=3694160723

You should get 5 (!) requests from all over the world in your webservers log. Not just outbound1.letsencrypt.org.

1 Like

I only included the first request for brevity, but I am getting five request. However, all of the request are from outbound1.letsencrypt.org. If I am supposed to be getting request from other servers, GeoIP blocking could definitely be the cause as I have limited access to my services to connections from the US only (we are a US based company and all the services are for employees).

HTTP Full Capture

3472  19.602232 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 48123 β†’ 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=3694160449 TSecr=0 WS=128
 3473  19.602304 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 80 β†’ 48123 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=3573006503 TSecr=3694160449
 3479  19.655643 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48123 β†’ 80 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=3694160502 TSecr=3573006503
 3480  19.656078 outbound1.letsencrypt.org β†’ vpn.delta-tech.com HTTP 337 GET /.well-known/acme-challenge/Z2cNYL4TmB_9JzjUB9nmXuOVhEH5XAM66lpp8gf4s7k HTTP/1.1 
 3481  19.656094 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 48123 [ACK] Seq=1 Ack=272 Win=65536 Len=0 TSval=3573006557 TSecr=3694160502
 3482  19.657342 vpn.delta-tech.com β†’ outbound1.letsencrypt.org HTTP 766 HTTP/1.1 301 Moved Permanently  (text/html)
 3483  19.657353 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 48123 [FIN, ACK] Seq=701 Ack=272 Win=65792 Len=0 TSval=3573006558 TSecr=3694160502
 3484  19.668834 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 48131 β†’ 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=3694160515 TSecr=0 WS=128
 3485  19.668869 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 80 β†’ 48131 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=3055903269 TSecr=3694160515
 3491  19.711253 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48123 β†’ 80 [ACK] Seq=272 Ack=701 Win=64128 Len=0 TSval=3694160558 TSecr=3573006558
 3492  19.722613 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48131 β†’ 80 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=3694160569 TSecr=3055903269
 3493  19.723123 outbound1.letsencrypt.org β†’ vpn.delta-tech.com HTTP 339 GET /.well-known/acme-challenge/5o6xHruvq5ySEGqavzTZ-sJZ1y1nDMs5KKUBC6SxF7I HTTP/1.1 
 3494  19.723142 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 48131 [ACK] Seq=1 Ack=274 Win=65536 Len=0 TSval=3055903324 TSecr=3694160569
 3495  19.724399 vpn.delta-tech.com β†’ outbound1.letsencrypt.org HTTP 768 HTTP/1.1 301 Moved Permanently  (text/html)
 3496  19.724410 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 48131 [FIN, ACK] Seq=703 Ack=274 Win=65792 Len=0 TSval=3055903325 TSecr=3694160569
 3498  19.758119 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48123 β†’ 80 [ACK] Seq=272 Ack=702 Win=64128 Len=0 TSval=3694160605 TSecr=3573006558
 3499  19.760152 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48123 β†’ 80 [FIN, ACK] Seq=272 Ack=702 Win=64128 Len=0 TSval=3694160607 TSecr=3573006558
 3500  19.760168 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 48123 [ACK] Seq=702 Ack=273 Win=65792 Len=0 TSval=3573006661 TSecr=3694160607
 3503  19.760415 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 38907 β†’ 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=2502722490 TSecr=0 WS=128
 3504  19.760430 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 80 β†’ 38907 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=2437567068 TSecr=2502722490
 3505  19.778051 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48131 β†’ 80 [ACK] Seq=274 Ack=703 Win=64128 Len=0 TSval=3694160624 TSecr=3055903325
 3507  19.799220 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 48139 β†’ 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=3694160646 TSecr=0 WS=128
 3508  19.799284 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 80 β†’ 48139 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=3045101956 TSecr=3694160646
 3510  19.814439 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 38907 β†’ 80 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=2502722544 TSecr=2437567068
 3511  19.814884 outbound1.letsencrypt.org β†’ vpn.delta-tech.com HTTP 337 GET /.well-known/acme-challenge/94FtlA0cmiin8VzHlon1_KC8NbSA35T-H7Emd6caT3I HTTP/1.1 
 3512  19.814915 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 38907 [ACK] Seq=1 Ack=272 Win=65536 Len=0 TSval=2437567122 TSecr=2502722544
 3515  19.816344 vpn.delta-tech.com β†’ outbound1.letsencrypt.org HTTP 766 HTTP/1.1 301 Moved Permanently  (text/html)
 3516  19.816357 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 38907 [FIN, ACK] Seq=701 Ack=272 Win=65792 Len=0 TSval=2437567124 TSecr=2502722544
 3520  19.821958 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48131 β†’ 80 [ACK] Seq=274 Ack=704 Win=64128 Len=0 TSval=3694160669 TSecr=3055903325
 3521  19.844089 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48131 β†’ 80 [FIN, ACK] Seq=274 Ack=704 Win=64128 Len=0 TSval=3694160691 TSecr=3055903325
 3522  19.844155 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 48131 [ACK] Seq=704 Ack=275 Win=65792 Len=0 TSval=3055903445 TSecr=3694160691
 3525  19.853212 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48139 β†’ 80 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=3694160700 TSecr=3045101956
 3526  19.853633 outbound1.letsencrypt.org β†’ vpn.delta-tech.com HTTP 340 GET /.well-known/acme-challenge/pQfARzxX60sSEAO5zPT7jP4A3tJd1SPr4JQwOeSovI4 HTTP/1.1 
 3527  19.853660 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 48139 [ACK] Seq=1 Ack=275 Win=65536 Len=0 TSval=3045102010 TSecr=3694160700
 3528  19.854890 vpn.delta-tech.com β†’ outbound1.letsencrypt.org HTTP 769 HTTP/1.1 301 Moved Permanently  (text/html)
 3529  19.854901 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 48139 [FIN, ACK] Seq=704 Ack=275 Win=65792 Len=0 TSval=3045102011 TSecr=3694160700
 3530  19.869799 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 38907 β†’ 80 [ACK] Seq=272 Ack=701 Win=64128 Len=0 TSval=2502722599 TSecr=2437567124
 3549  19.908678 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48139 β†’ 80 [ACK] Seq=275 Ack=704 Win=64128 Len=0 TSval=3694160755 TSecr=3045102011
 3550  19.912081 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 38907 β†’ 80 [ACK] Seq=272 Ack=702 Win=64128 Len=0 TSval=2502722642 TSecr=2437567124
 3551  19.913552 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 38907 β†’ 80 [FIN, ACK] Seq=272 Ack=702 Win=64128 Len=0 TSval=2502722643 TSecr=2437567124
 3552  19.913563 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 38907 [ACK] Seq=702 Ack=273 Win=65792 Len=0 TSval=2437567221 TSecr=2502722643
 3560  19.950044 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48139 β†’ 80 [ACK] Seq=275 Ack=705 Win=64128 Len=0 TSval=3694160797 TSecr=3045102011
 3561  19.953172 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 48139 β†’ 80 [FIN, ACK] Seq=275 Ack=705 Win=64128 Len=0 TSval=3694160800 TSecr=3045102011
 3562  19.953186 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 80 β†’ 48139 [ACK] Seq=705 Ack=276 Win=65792 Len=0 TSval=3045102110 TSecr=3694160800

HTTPS Full Capture

3501  19.760358 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 61327 β†’ 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=3694160607 TSecr=0 WS=128
 3502  19.760406 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 443 β†’ 61327 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=3459792963 TSecr=3694160607
 3509  19.814353 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61327 β†’ 443 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=3694160661 TSecr=3459792963
 3513  19.815066 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1 327 Client Hello (SNI=imap.delta-tech.com)
 3514  19.815087 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61327 [ACK] Seq=1 Ack=262 Win=65536 Len=0 TSval=3459793018 TSecr=3694160661
 3517  19.821509 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 1514 Server Hello, Change Cipher Spec, Application Data
 3518  19.821542 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 1514 Application Data
 3519  19.821553 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 366 Application Data, Application Data
 3523  19.844201 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 61337 β†’ 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=3694160691 TSecr=0 WS=128
 3524  19.844289 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 443 β†’ 61337 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=723317719 TSecr=3694160691
 3531  19.875541 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61327 β†’ 443 [ACK] Seq=262 Ack=1449 Win=64128 Len=0 TSval=3694160722 TSecr=3459793024
 3532  19.875555 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61327 β†’ 443 [ACK] Seq=262 Ack=2897 Win=63616 Len=0 TSval=3694160722 TSecr=3459793024
 3533  19.875563 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61327 β†’ 443 [ACK] Seq=262 Ack=3197 Win=63360 Len=0 TSval=3694160722 TSecr=3459793024
 3534  19.876228 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 146 Change Cipher Spec, Application Data
 3535  19.876244 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61327 [ACK] Seq=3197 Ack=342 Win=65792 Len=0 TSval=3459793079 TSecr=3694160723
 3536  19.876254 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 467 Application Data
 3537  19.876261 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61327 [ACK] Seq=3197 Ack=743 Win=65408 Len=0 TSval=3459793079 TSecr=3694160723
 3538  19.877036 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 353 Application Data
 3539  19.877215 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 353 Application Data
 3540  19.877534 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 677 Application Data
 3541  19.877658 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 90 Application Data
 3542  19.877798 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61327 [FIN, ACK] Seq=4406 Ack=743 Win=65792 Len=0 TSval=3459793080 TSecr=3694160723
 3543  19.897572 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61337 β†’ 443 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=3694160744 TSecr=723317719
 3544  19.898119 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1 329 Client Hello (SNI=mail01.delta-tech.com)
 3545  19.898131 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61337 [ACK] Seq=1 Ack=264 Win=65536 Len=0 TSval=723317773 TSecr=3694160745
 3546  19.905053 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 1514 Server Hello, Change Cipher Spec, Application Data
 3547  19.905139 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 1514 Application Data
 3548  19.905148 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 366 Application Data, Application Data
 3553  19.913712 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 42585 β†’ 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=2502722643 TSecr=0 WS=128
 3554  19.913745 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 443 β†’ 42585 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=3873446467 TSecr=2502722643
 3555  19.931093 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61327 β†’ 443 [ACK] Seq=743 Ack=4407 Win=64128 Len=0 TSval=3694160778 TSecr=3459793080
 3556  19.931110 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 90 Application Data
 3557  19.931121 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61327 [ACK] Seq=4407 Ack=767 Win=65792 Len=0 TSval=3459793134 TSecr=3694160778
 3558  19.931225 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61327 β†’ 443 [FIN, ACK] Seq=767 Ack=4407 Win=64128 Len=0 TSval=3694160778 TSecr=3459793080
 3559  19.931236 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61327 [ACK] Seq=4407 Ack=768 Win=65792 Len=0 TSval=3459793134 TSecr=3694160778
 3563  19.953371 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 74 61351 β†’ 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=3694160800 TSecr=0 WS=128
 3564  19.953399 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 74 443 β†’ 61351 [SYN, ACK] Seq=0 Ack=1 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM TSval=673921139 TSecr=3694160800
 3565  19.959061 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61337 β†’ 443 [ACK] Seq=264 Ack=1449 Win=64128 Len=0 TSval=3694160806 TSecr=723317780
 3566  19.959073 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61337 β†’ 443 [ACK] Seq=264 Ack=2897 Win=63616 Len=0 TSval=3694160806 TSecr=723317780
 3567  19.959080 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61337 β†’ 443 [ACK] Seq=264 Ack=3197 Win=63360 Len=0 TSval=3694160806 TSecr=723317780
 3568  19.959777 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 146 Change Cipher Spec, Application Data
 3569  19.959788 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61337 [ACK] Seq=3197 Ack=344 Win=65792 Len=0 TSval=723317834 TSecr=3694160806
 3570  19.959799 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 471 Application Data
 3571  19.959806 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61337 [ACK] Seq=3197 Ack=749 Win=65408 Len=0 TSval=723317834 TSecr=3694160806
 3572  19.960527 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 353 Application Data
 3573  19.960716 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 353 Application Data
 3574  19.960971 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 677 Application Data
 3575  19.961125 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 90 Application Data
 3576  19.961255 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61337 [FIN, ACK] Seq=4406 Ack=749 Win=65792 Len=0 TSval=723317836 TSecr=3694160806
 3577  19.967270 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 42585 β†’ 443 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=2502722697 TSecr=3873446467
 3578  19.968017 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1 327 Client Hello (SNI=smtp.delta-tech.com)
 3579  19.968032 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 42585 [ACK] Seq=1 Ack=262 Win=65536 Len=0 TSval=3873446522 TSecr=2502722697
 3580  19.973356 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 1514 Server Hello, Change Cipher Spec, Application Data
 3581  19.973439 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 1514 Application Data
 3582  19.973450 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 366 Application Data, Application Data
 3583  20.007610 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61351 β†’ 443 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=3694160854 TSecr=673921139
 3584  20.008271 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1 330 Client Hello (SNI=webmail.delta-tech.com)
 3585  20.008299 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61351 [ACK] Seq=1 Ack=265 Win=65536 Len=0 TSval=673921194 TSecr=3694160855
 3586  20.013726 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 1514 Server Hello, Change Cipher Spec, Application Data
 3587  20.013789 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 1514 Application Data
 3588  20.013794 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 366 Application Data, Application Data
 3589  20.014750 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61337 β†’ 443 [ACK] Seq=749 Ack=4407 Win=64128 Len=0 TSval=3694160861 TSecr=723317835
 3590  20.015017 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 90 Application Data
 3591  20.015037 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61337 [ACK] Seq=4407 Ack=773 Win=65792 Len=0 TSval=723317890 TSecr=3694160861
 3592  20.015046 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61337 β†’ 443 [FIN, ACK] Seq=773 Ack=4407 Win=64128 Len=0 TSval=3694160862 TSecr=723317835
 3593  20.015054 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61337 [ACK] Seq=4407 Ack=774 Win=65792 Len=0 TSval=723317890 TSecr=3694160862
 3594  20.027110 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 42585 β†’ 443 [ACK] Seq=262 Ack=1449 Win=64128 Len=0 TSval=2502722757 TSecr=3873446527
 3595  20.027134 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 42585 β†’ 443 [ACK] Seq=262 Ack=2897 Win=63616 Len=0 TSval=2502722757 TSecr=3873446527
 3596  20.027143 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 42585 β†’ 443 [ACK] Seq=262 Ack=3197 Win=63360 Len=0 TSval=2502722757 TSecr=3873446527
 3597  20.028338 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 146 Change Cipher Spec, Application Data
 3598  20.028357 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 42585 [ACK] Seq=3197 Ack=342 Win=65792 Len=0 TSval=3873446582 TSecr=2502722758
 3599  20.028513 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 467 Application Data
 3600  20.028524 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 42585 [ACK] Seq=3197 Ack=743 Win=65408 Len=0 TSval=3873446582 TSecr=2502722758
 3601  20.029073 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 353 Application Data
 3602  20.029245 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 353 Application Data
 3603  20.029521 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 677 Application Data
 3604  20.029678 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 90 Application Data
 3605  20.029820 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 42585 [FIN, ACK] Seq=4406 Ack=743 Win=65792 Len=0 TSval=3873446583 TSecr=2502722758
 3608  20.067424 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61351 β†’ 443 [ACK] Seq=265 Ack=1449 Win=64128 Len=0 TSval=3694160914 TSecr=673921199
 3609  20.067471 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61351 β†’ 443 [ACK] Seq=265 Ack=3197 Win=62848 Len=0 TSval=3694160914 TSecr=673921199
 3610  20.068209 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 146 Change Cipher Spec, Application Data
 3611  20.068233 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61351 [ACK] Seq=3197 Ack=345 Win=65792 Len=0 TSval=673921254 TSecr=3694160915
 3612  20.068247 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 473 Application Data
 3613  20.068256 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61351 [ACK] Seq=3197 Ack=752 Win=65408 Len=0 TSval=673921254 TSecr=3694160915
 3614  20.069021 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 353 Application Data
 3615  20.069260 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 353 Application Data
 3616  20.069509 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 677 Application Data
 3617  20.069633 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TLSv1.3 90 Application Data
 3618  20.069775 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61351 [FIN, ACK] Seq=4406 Ack=752 Win=65792 Len=0 TSval=673921255 TSecr=3694160915
 3625  20.082789 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 42585 β†’ 443 [ACK] Seq=743 Ack=4406 Win=64128 Len=0 TSval=2502722812 TSecr=3873446583
 3626  20.082823 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 90 Application Data
 3627  20.082837 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 [TCP Retransmission] 443 β†’ 42585 [FIN, ACK] Seq=4406 Ack=767 Win=65792 Len=0 TSval=3873446636 TSecr=2502722812
 3628  20.082844 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 42585 β†’ 443 [FIN, ACK] Seq=767 Ack=4406 Win=64128 Len=0 TSval=2502722812 TSecr=3873446583
 3629  20.082854 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 [TCP Retransmission] 443 β†’ 42585 [FIN, ACK] Seq=4406 Ack=768 Win=65792 Len=0 TSval=3873446636 TSecr=2502722812
 3630  20.083997 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 42585 β†’ 443 [ACK] Seq=768 Ack=4407 Win=64128 Len=0 TSval=2502722813 TSecr=3873446583
 3641  20.123088 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61351 β†’ 443 [ACK] Seq=752 Ack=4407 Win=64128 Len=0 TSval=3694160970 TSecr=673921254
 3642  20.123292 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TLSv1.3 90 Application Data
 3643  20.123311 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61351 [ACK] Seq=4407 Ack=776 Win=65792 Len=0 TSval=673921309 TSecr=3694160970
 3644  20.123320 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 66 61351 β†’ 443 [FIN, ACK] Seq=776 Ack=4407 Win=64128 Len=0 TSval=3694160970 TSecr=673921254
 3645  20.123370 vpn.delta-tech.com β†’ outbound1.letsencrypt.org TCP 66 443 β†’ 61351 [ACK] Seq=4407 Ack=777 Win=65792 Len=0 TSval=673921309 TSecr=3694160970
 3652  20.137088 outbound1.letsencrypt.org β†’ vpn.delta-tech.com TCP 78 [TCP Dup ACK 3630#1] 42585 β†’ 443 [ACK] Seq=768 Ack=4407 Win=64128 Len=0 TSval=2502722867 TSecr=3873446636 SLE=4406 SRE=4407

Please see the post from Peter above. You're geoblocking everything but the US, which is giving you trouble.

2 Likes

You should not be doing geo-blocking if using HTTP-01 validation method. Otherwise, you should switch validation to DNS-01.

2 Likes

Geoblocking HTTP is unnecessary.
[when you follow best practice and redirect all HTTP to HTTPS]

2 Likes

SOLVED!

Thanks for all the help. It was GeoIP blocking that was causing the renewals to fail. Once I disabled GeoIP blocking, all challenges were successful.

1 Like

I have looked into DNS validation. Unfortunately, I am currently using GoDaddy as my DNS provider and they appear to be supported by certbot.

I had good effects using GitHub - joohoi/acme-dns: Limited DNS server with RESTful HTTP API to handle ACME DNS challenges easily and securely.
Author of this project runs a free service, which you can CNAME to, and then use the API from that service, instead of GoDaddy.

All of my websites do redirect to HTTPS. I do GeoIP blocking for internal services mostly to follow the principal of least privilege. We are a small US based business, so there is no one outside the US that has any legitimate reason to connect to any of our internal services.

When I implemented GeoIP blocking intrusion attempts for SSH went from around 120 attempts/day to about 15 attempts/day. In addition, GeoIP blocking has considerably reduced the amount or spam/phishing emails we receive.

There is a third party DNS plugin for GoDaddy out there, but I believe they have modified their API so that not everyone has access to it any longer. See GoDaddy no longer allows API access to clients (e.g. for DNS-based cert renewal) if you have less than 50 domains for more info.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.