Error getting validation data", "status": 400

mgdaniel · March 14, 2024, 6:02pm

(apologies as I edit to clean this up ... my first post)

My domain is: mail.vsm.com.au

I ran this command:

mcr cgi_exe:wucme.exe --uacme --force --verbose issue mail.vsm.com.au

It produced this output:

$ mcr cgi_exe:wucme.exe --uacme --force --verbose issue mail.vsm.com.au
2024-03-15 04:08:54.40: version WUCME IA64-1.1.8 (1.1.2) (OpenSSL 3.0.7 9 Nov 2022) starting
2024-03-15 04:08:54.40: loading key from /wasd_root/local/wucme_k_account.pem
2024-03-15 04:08:54.45: loading key from /wasd_root/local/wucme_k_mail_vsm_com_au.pem
2024-03-15 04:08:54.48: checking existence and expiration of /wasd_root/local/wucme_c_mail_vsm_com_au.pem
2024-03-15 04:08:54.51: /wasd_root/local/wucme_c_mail_vsm_com_au.pem expires in 16 days
2024-03-15 04:08:54.51: /wasd_root/local/wucme_c_mail_vsm_com_au.pem is due for renewal
2024-03-15 04:08:54.51: fetching directory at https://acme-v02.api.letsencrypt.org/directory
2024-03-15 04:08:58.71: retrieving account at https://acme-v02.api.letsencrypt.org/acme/new-acct
8< snip of sensitive data 8<
2024-03-15 04:09:11.60: polling challenge status at https://acme-v02.api.letsencrypt.org/acme/chall-v3/326275863427/SQ7PVQ
2024-03-15 04:09:12.79: challenge https://acme-v02.api.letsencrypt.org/acme/chall-v3/326275863427/SQ7PVQ failed with status invalid
2024-03-15 04:09:12.79: the server reported the following error:
{
    "type": "urn:ietf:params:acme:error:connection",
    "detail": "During secondary validation: 119.252.17.13: Fetching http://mail.vsm.com.au/.well-known/acme-challenge/l6Z-lhDjeP80iP
eZ_iKNVb7sd6L24WTJcMpxKHTLPhc: Error getting validation data",
    "status": 400
}
2024-03-15 04:09:12.79: failed to authorize order at https://acme-v02.api.letsencrypt.org/acme/order/17298823/252306525507

This does not appear to be dns/firewall/etc issue as the web server is observed receiving and processing the challenge request and response.

|04:09:09.34 SERVICE  1779 000002 CONNECT    VIRTUAL mail.vsm.com.au:80|
|04:09:09.34 REQUEST  4442 000002 REQUEST    GET /.well-known/acme-challenge/l6Z-lhDjeP80iPeZ_8< security snip 8<|
8< snip 8<
|04:09:09.37 NET      2696 000002 RES-HEADER HEADER 275 bytes|
HTTP/1.1 200 OK
Server: HTTPd-WASD/12.2.0 OpenVMS/IA64 SSL
Date: Thu, 14 Mar 2024 17:39:09 GMT
Accept-Ranges: bytes
Accept-Encoding: gzip, deflate
Content-Length: 87
Connection: close
Cache-Control: no-cache, no-store, must-revalidate
Pragma: no-cache
Expires: 0

|04:09:09.37 NET      2839 000002 RES-BODY   BODY 87 bytes|
6C365A2D 6C68446A 65503830 6950655A 5F694B4E 56623773 64364C32 3457544A l6Z-lhDjeP80iPeZ_iKNVb7sd6L24WTJ
8< 32 bytes snipped for sensitivity 8<
4A6D5A55 73744737 66666A42 76684E6D 645F472D 4C7338                     JmZUstG7ffjBvhNmd_G-Ls8
8< snip 8<

As can be seen from crt.sh this has been working up until recently and for other sites continues to work. Seems to be just be VSM certs.

My web server is (include version): WASD v12.2.0

The operating system my web server runs on is (include version): VMS V8.4-2L3

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): WUCME IA64-1.1.8 (1.1.2) (OpenSSL 3.0.7 9 Nov 2022) ... this is essentially uACME v1.1.2 and has been running OK since 2016-12-31 (from crt.sh)

petercooperjr · March 14, 2024, 6:14pm

How many requests are you seeing? There should be at least three, and they're working on adding even more.

mgdaniel · March 14, 2024, 6:19pm

Just the one.

petercooperjr · March 14, 2024, 6:21pm

Then you (or something upstream of you) has something blocking some of the requests that Let's Encrypt is using to try to validate that you own the name.

mgdaniel · March 14, 2024, 6:31pm

Hmmm. Will pass this on to the site admin. He better understands the hosting setup. Thanks Peter.

webprofusion · March 15, 2024, 6:00am

I'd also check that your (custom) challenge response service is not simply flushing used values once they've been read once or twice, I'd say they need to stick around for a minute or so before cleaning them up. Your service returns http status code 200 (OK), instead of a 404 or other error when it can't find a matching challenge or part thereof:

curl http://mail.vsm.com.au/.well-known/acme-challenge/l6Z-lhDjeP80iP -v

*   Trying 119.252.17.13:80...
* Connected to mail.vsm.com.au (119.252.17.13) port 80 (#0)
> GET /.well-known/acme-challenge/l6Z-lhDjeP80iP HTTP/1.1
> Host: mail.vsm.com.au
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: HTTPd-WASD/12.2.0 OpenVMS/IA64 SSL
< Date: Fri, 15 Mar 2024 05:58:01 GMT
< Accept-Ranges: bytes
< Accept-Encoding: gzip, deflate
< Content-Type: text/plain; charset=ISO-8859-1
< Transfer-Encoding: chunked
< Cache-Control: no-cache, no-store, must-revalidate
< Pragma: no-cache
< Expires: 0
<
Challenge received but no such token.

mgdaniel · March 15, 2024, 9:46am

The challenge response is generated dynamically with each challenge request. There are no "used values" stored away or floating about. The response header goes out of its way to ensure caching does not store. It certainly appears to be a better option for a non-success response (as the above 'curl' request) to return 404 rather than 200. However that is a refinement (I'll implement in a future release), and the current implementation has been working for some time now. Thanks for your input.

mgdaniel · March 15, 2024, 11:07am

Have passed this on to the site admin. He suggested his DNS tools showed not an issue with (global) host name space. And will look at system / up-stream issues over the weekend. Thanks for your input.

mgdaniel · March 16, 2024, 3:50am

My fault! A couple of weeks ago the site was extremely sluggish and upon investigation found repeated bursts of activity grinding away at resources. Looking at the requests there was nothing distinguishing them and no bot identified in the user-agent. Just grinding it into the ground. Ham-fisted DOS? Trawler loose from its moorings? Not obvious.

So I took out my trusty sledgehammer and rejected anything from *amazonaws*. Grinding immediately stopped. Didn't think any more about it until a couple of hours ago. Removed the rule. Voila!

|13:18:40.27 NET      2202 000015 CONNECT    ACCEPTED outbound1b.letsencrypt.org,43751 on http://www.vsm.com.au,80 (119.252.17.13) BG44236:|
8< snip 8<
|13:18:40.87 NET      2202 000016 CONNECT    ACCEPTED ec2-18-216-65-86.us-east-2.compute.amazonaws.com,41306 on http://www.vsm.com.au,80 (119.252.17.13) BG44246:|
8< snip 8<
|13:18:41.58 NET      2202 000018 CONNECT    ACCEPTED ec2-18-237-231-133.us-west-2.compute.amazonaws.com,34448 on http://www.vsm.com.au,80 (119.252.17.13) BG44249:|

Three validations and happily a renewed certificate.

I will write-up the hazards of using oversize tools for cracking soft-shell almonds as an object lesson for our user community.

Thanks for your (plural) inputs.

system · April 15, 2024, 3:50am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.