Failing to renew certificates on step 6

Help
1

My domain is: https://generalgrains.com/

I ran this command: v-add-letsencrypt-domain admin generalgrains.com

It produced this output: Error: Let's Encrypt finalize bad status 403
--- Sending CSR to finalize order / STEP 6 ---
[Mon Oct 13 09:10:43 MDT 2025] : answer=HTTP/2 403
server: nginx
date: Mon, 13 Oct 2025 15:10:43 GMT
content-type: application/problem+json
content-length: 152
boulder-requester: 1450010146
cache-control: public, max-age=0, no-cache
{
"type": "urn:ietf:params:acme:error:orderNotReady",
"detail": "Order's status ("invalid") is not acceptable for finalization",
"status": 403
}

My web server is (include version): nginx version: nginx/1.18.0 & Server version: Apache/2.4.25 (Debian)

The operating system my web server runs on is (include version): Debian GNU/Linux 9

My hosting provider, if applicable, is: Self-Hosted

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): Yes - vesta control panel

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

it seems to go through the whole proccess with steps 1-5 succeding but failing on step 6
[Mon Oct 13 09:10:37 MDT 2025] : --- Requesting nonce / STEP 1 ---
[Mon Oct 13 09:10:37 MDT 2025] : curl -s -I "https://acme-v02.api.letsencrypt.org/directory"
[Mon Oct 13 09:10:37 MDT 2025] : answer=HTTP/2 200

[Mon Oct 13 09:10:37 MDT 2025] : --- Placing new order / STEP 2 ---
[Mon Oct 13 09:10:37 MDT 2025] : payload={"identifiers":[{"type":"dns","value":"generalgrains.com"}]}
[Mon Oct 13 09:10:37 MDT 2025] : answer=HTTP/2 201

[Mon Oct 13 09:10:37 MDT 2025] : --- Requesting authorization token / STEP 3 ---
[Mon Oct 13 09:10:37 MDT 2025] : answer=HTTP/2 200

[Mon Oct 13 09:10:37 MDT 2025] : --- Configuring challenge / STEP 4 ---
[Mon Oct 13 09:10:37 MDT 2025] : wildcard=
[Mon Oct 13 09:10:37 MDT 2025] : in
[Mon Oct 13 09:10:37 MDT 2025] : v-restart-proxy

[Mon Oct 13 09:10:37 MDT 2025] : --- Requesting ACME validation / STEP 5 ---
[Mon Oct 13 09:10:37 MDT 2025] : validation_check=
[Mon Oct 13 09:10:37 MDT 2025] : - Doing pol check on status
[Mon Oct 13 09:10:38 MDT 2025] : answer=HTTP/2 200

Any help or assistance is appreciated. Note: this is happing for multiple sites on this machine.

2

Would you explain how these two web servers are used? I see nginx responding to HTTP (port 80) requests. Is that what I should see?

Is there any more info from that log? Did you remove any parts?

Because step 6, the "finalize" failing, means something in the prior steps failed. The error is that the "order" created in step 2 is in an "invalid" state. It should be in a "ready" state for the "finalize" request to succeed. In fact, your ACME Client shouldn't even try the finalize if it does not see a suitable order state (like "ready") when it polls the status.

The "200" http responses in steps 1-5 just mean the HTTPS request from your machine to the Let's Encrypt server succeeded. But, the data in that reply is what is most important. Likely the challenge / authorization steps failed. If those steps had succeeded the "order" would be "ready".

3

Yes nginx is what should be used.

Sure, here's the full output from the logs

[Mon Oct 13 09:10:37 MDT 2025] : v-add-letsencrypt-domain generalgrains.com
[Mon Oct 13 09:10:37 MDT 2025] : v-add-letsencrypt-user admin
[Mon Oct 13 09:10:37 MDT 2025] : result: 0
[Mon Oct 13 09:10:37 MDT 2025] : --- Requesting nonce / STEP 1 ---
[Mon Oct 13 09:10:37 MDT 2025] : curl -s -I "https://acme-v02.api.letsencrypt.org/directory"
[Mon Oct 13 09:10:37 MDT 2025] : answer=HTTP/2 200
server: nginx
date: Mon, 13 Oct 2025 15:10:37 GMT
content-type: application/json
content-length: 1065
cache-control: public, max-age=0, no-cache
replay-nonce: D3QZ_ywxaqEfgUAlQcg6F01PacEECybKGd5Z9iPB1AJ12iV66UQ
x-frame-options: DENY
strict-transport-security: max-age=604800

[Mon Oct 13 09:10:37 MDT 2025] : nonce=D3QZ_ywxaqEfgUAlQcg6F01PacEECybKGd5Z9iPB1AJ12iV66UQ
[Mon Oct 13 09:10:37 MDT 2025] : status=200
[Mon Oct 13 09:10:37 MDT 2025] : --- Placing new order / STEP 2 ---
[Mon Oct 13 09:10:37 MDT 2025] : payload={"identifiers":[{"type":"dns","value":"generalgrains.com"}]}
[Mon Oct 13 09:10:37 MDT 2025] : query_le_v2 "https://acme-v02.api.letsencrypt.org/acme/new-order" "{"identifiers":[{"type":"dns","value":"generalgrains.com"}]}" "D3QZ_ywxaqEfgUAlQcg6F01PacEECybKGd5Z9iPB1AJ12iV66UQ"
[Mon Oct 13 09:10:37 MDT 2025] : answer=HTTP/2 201
server: nginx
date: Mon, 13 Oct 2025 15:10:37 GMT
content-type: application/json
content-length: 351
boulder-requester: 1450010146
cache-control: public, max-age=0, no-cache
link: https://acme-v02.api.letsencrypt.org/directory;rel="index"
location: https://acme-v02.api.letsencrypt.org/acme/order/1450010146/437637773571
replay-nonce: P2omldGHbBOfqZHEsQR_Yf9OechuJ5K91aRz1lwDuoK8nijTfqA
x-frame-options: DENY
strict-transport-security: max-age=604800

{
"status": "pending",
"expires": "2025-10-20T15:10:37Z",
"identifiers": [
{
"type": "dns",
"value": "generalgrains.com"
}
],
"authorizations": [
"https://acme-v02.api.letsencrypt.org/acme/authz/1450010146/597228055291"
],
"finalize": "https://acme-v02.api.letsencrypt.org/acme/finalize/1450010146/437637773571"
}
[Mon Oct 13 09:10:37 MDT 2025] : nonce=P2omldGHbBOfqZHEsQR_Yf9OechuJ5K91aRz1lwDuoK8nijTfqA
[Mon Oct 13 09:10:37 MDT 2025] : authz=https://acme-v02.api.letsencrypt.org/acme/authz/1450010146/597228055291
[Mon Oct 13 09:10:37 MDT 2025] : finalize=https://acme-v02.api.letsencrypt.org/acme/finalize/1450010146/437637773571
[Mon Oct 13 09:10:37 MDT 2025] : status=201
[Mon Oct 13 09:10:37 MDT 2025] : --- Requesting authorization token / STEP 3 ---
[Mon Oct 13 09:10:37 MDT 2025] : for auth=https://acme-v02.api.letsencrypt.org/acme/authz/1450010146/597228055291
[Mon Oct 13 09:10:37 MDT 2025] : query_le_v2 "https://acme-v02.api.letsencrypt.org/acme/authz/1450010146/597228055291" "" "P2omldGHbBOfqZHEsQR_Yf9OechuJ5K91aRz1lwDuoK8nijTfqA"
[Mon Oct 13 09:10:37 MDT 2025] : answer=HTTP/2 200
server: nginx
date: Mon, 13 Oct 2025 15:10:37 GMT
content-type: application/json
content-length: 825
boulder-requester: 1450010146
cache-control: public, max-age=0, no-cache
link: https://acme-v02.api.letsencrypt.org/directory;rel="index"
replay-nonce: S5Fh2depGn3AXl3DDAN9Ux2O6qWSIzJ9JYm37kYRZL6B4O_fEZw
x-frame-options: DENY
strict-transport-security: max-age=604800

{
"identifier": {
"type": "dns",
"value": "generalgrains.com"
},
"status": "pending",
"expires": "2025-10-20T15:10:37Z",
"challenges": [
{
"type": "tls-alpn-01",
"url": "https://acme-v02.api.letsencrypt.org/acme/chall/1450010146/597228055291/ZVO28A",
"status": "pending",
"token": "2OygqlRMI_pQbmP0cu73me9H05dKIP7U2CFHNQt8NFk"
},
{
"type": "dns-01",
"url": "https://acme-v02.api.letsencrypt.org/acme/chall/1450010146/597228055291/BsJLVw",
"status": "pending",
"token": "2OygqlRMI_pQbmP0cu73me9H05dKIP7U2CFHNQt8NFk"
},
{
"type": "http-01",
"url": "https://acme-v02.api.letsencrypt.org/acme/chall/1450010146/597228055291/tiSPwA",
"status": "pending",
"token": "2OygqlRMI_pQbmP0cu73me9H05dKIP7U2CFHNQt8NFk"
}
]
}
[Mon Oct 13 09:10:37 MDT 2025] : url=https://acme-v02.api.letsencrypt.org/acme/chall/1450010146/597228055291/tiSPwA
[Mon Oct 13 09:10:37 MDT 2025] : token=2OygqlRMI_pQbmP0cu73me9H05dKIP7U2CFHNQt8NFk
[Mon Oct 13 09:10:37 MDT 2025] : nonce=S5Fh2depGn3AXl3DDAN9Ux2O6qWSIzJ9JYm37kYRZL6B4O_fEZw
[Mon Oct 13 09:10:37 MDT 2025] : status=200
[Mon Oct 13 09:10:37 MDT 2025] : --- Configuring challenge / STEP 4 ---
[Mon Oct 13 09:10:37 MDT 2025] : wildcard=
[Mon Oct 13 09:10:37 MDT 2025] : in /home/admin/conf/web/nginx.generalgrains.com.conf_letsencrypt we put: jlWAKcE2wtaUED-CphA60N1MhES1s_ZJHlrKbB8TRw0
[Mon Oct 13 09:10:37 MDT 2025] : v-restart-proxy
[Mon Oct 13 09:10:37 MDT 2025] : --- Requesting ACME validation / STEP 5 ---
[Mon Oct 13 09:10:37 MDT 2025] : validation_check=
[Mon Oct 13 09:10:37 MDT 2025] : - Doing pol check on status
[Mon Oct 13 09:10:37 MDT 2025] : query_le_v2 "https://acme-v02.api.letsencrypt.org/acme/chall/1450010146/597228055291/tiSPwA" "{}" "S5Fh2depGn3AXl3DDAN9Ux2O6qWSIzJ9JYm37kYRZL6B4O_fEZw"
[Mon Oct 13 09:10:38 MDT 2025] : answer=HTTP/2 200
server: nginx
date: Mon, 13 Oct 2025 15:10:38 GMT
content-type: application/json
content-length: 195
boulder-requester: 1450010146
cache-control: public, max-age=0, no-cache
link: https://acme-v02.api.letsencrypt.org/directory;rel="index"
link: https://acme-v02.api.letsencrypt.org/acme/authz/1450010146/597228055291;rel="up"
location: https://acme-v02.api.letsencrypt.org/acme/chall/1450010146/597228055291/tiSPwA
replay-nonce: P2omldGHG1qVGTS-671oJWwhfMqS7HmVJi0UCPG5J-ecqCM31PY
x-frame-options: DENY
strict-transport-security: max-age=604800

{
"type": "http-01",
"url": "https://acme-v02.api.letsencrypt.org/acme/chall/1450010146/597228055291/tiSPwA",
"status": "pending",
"token": "2OygqlRMI_pQbmP0cu73me9H05dKIP7U2CFHNQt8NFk"
}
[Mon Oct 13 09:10:38 MDT 2025] : validation=https://acme-v02.api.letsencrypt.org/acme/chall/1450010146/597228055291/tiSPwA
[Mon Oct 13 09:10:38 MDT 2025] : nonce=P2omldGHG1qVGTS-671oJWwhfMqS7HmVJi0UCPG5J-ecqCM31PY
[Mon Oct 13 09:10:38 MDT 2025] : status=200
[Mon Oct 13 09:10:38 MDT 2025] : sleep 4 (i=2)
[Mon Oct 13 09:10:43 MDT 2025] : --- Sending CSR to finalize order / STEP 6 ---
[Mon Oct 13 09:10:43 MDT 2025] : query_le_v2 "https://acme-v02.api.letsencrypt.org/acme/finalize/1450010146/437637773571" "{"csr":hidden for privacy"
[Mon Oct 13 09:10:43 MDT 2025] : answer=HTTP/2 403
server: nginx
date: Mon, 13 Oct 2025 15:10:43 GMT
content-type: application/problem+json
content-length: 152
boulder-requester: 1450010146
cache-control: public, max-age=0, no-cache
link: https://acme-v02.api.letsencrypt.org/directory;rel="index"
replay-nonce: P2omldGHrL-3w2a6Qx8OEQtWueakOsTX91bYLSVZ9DqjNyL9Bi8

{
"type": "urn:ietf:params:acme:error:orderNotReady",
"detail": "Order's status ("invalid") is not acceptable for finalization",
"status": 403
}
[Mon Oct 13 09:10:43 MDT 2025] : nonce=P2omldGHrL-3w2a6Qx8OEQtWueakOsTX91bYLSVZ9DqjNyL9Bi8
[Mon Oct 13 09:10:43 MDT 2025] : status=403
[Mon Oct 13 09:10:43 MDT 2025] : certificate=
[Mon Oct 13 09:10:43 MDT 2025] : EXIT=Let's Encrypt finalize bad status 403

4

Well, that program's log doesn't make it easy, but the challenge link here gives a reason

"error": {
    "type": "urn:ietf:params:acme:error:connection",
    "detail": "167.94.150.109: Fetching http://generalgrains.com/.well-known/acme-> challenge/2OygqlRMI_pQbmP0cu73me9H05dKIP7U2CFHNQt8NFk: Connection refused",
    "status": 400
  },

It looks like your site isn't accessible from some places in the world:

5

Yeah, the challenge failure "connection refused" is why the order is not ready for finalize.

It is a little strange in that a test from Let's Debug (https://letsdebug.net) works. And, none of the countries failing in check-host test host any Let's Encrypt data centers.

Is there some other firewall blocking specific IP addresses or ranges? The "connection refused" is likely something like that blocking the LE primary data center from reaching you. If one of the 4 secondary centers were failing we'd see a different error message.

Do you see any challenge requests in your nginx access log? They would have URI like this

/.well-known/acme-challenge/(token)

and a user-agent with "Let's Encrypt" in it.

6

Would that cause issues with the renewal process?
I would need to check with our security team about our blocking rules.
If the address is from the US I believe there should be no issue.

7

Let's Encrypt checks from multiple places around the world, to validate that the requester actually controls the name throughout the world.

8

I did not see any traffic in the access or error logs for nginx to the /.well-known/acme-challenge besides some malicious spam, and these logs go back over a year. Which is well before the last renew.

I can verify that there are no fw rules on the iptables blocking any traffic, I'll check with my security team to see if anything else on the main fw may be being blocked.
Something must have changed as we were having no issues for over a year or so.
I'll update this post if I find anything that may be useful or if the issue is resolved.

one question, if I try and renew will I hit rate limits?
That's something I was worried about, as on this server we have 9 different domains with their certs all being handled by vesta and Let's Encrypt.
Would there be a way to determine if this was a rate limit issue? Or would there be a different response entirely?

9

Then I would say that nginx isn't handling the Let's Encrypt challenges. You should be seeing 4 or 5 such requests each time a new cert is issued. The only exception is if you request a subsequent cert shortly after and Let's Encrypt authorization cache is still active. The cache varies depending on your "profile" with the default of 30 days.

But, looking back a year you should most definitely see some.

This needs to be figured out. I even used https://letsdebug.net for a couple tests to your domain earlier today so you should see those requests arriving in your nginx logs.

Are there multiple nginx systems? Could a firewall or similar device be using nginx that is responding rather than the one you looked at?

Maybe. Failed requests to production Let's Encrypt is 5 / hour. And, refills one every 12 minutes. Rate limits: Rate Limits - Let's Encrypt

If you can, use the Let's Encrypt staging system for test.

The error message is very clear if a rate limit. And, each such message includes a UTC time after which a retry may succeed.

10

Geo-Location blocking?

11

So after talking with Security, it seems we're Geo-Location blocking Russia, Iran, N. Korea, & China as this is where we receive the most malicious traffic/attacks from.
Is letsencrypt currently validating through one of these Geo-Locations?

12

No, none of the LE auth centers are in those countries.

And, as noted earlier ... from the error message it looks like the LE primary data center is not reaching you. That is located in the US.

Have you resolved why there are no log entries in the past year for an ACME Challenge?

13

I believe it's because of multiple log locations on the machine, still investigating as this vesta control panel is unknown to me, and was deployed by a former employee :sweat_smile: