The error is coming from your server, so I can't absolutely say -- it could redirect to an HTTPS url and then decide to give that error for fun if it wanted. But it is almost surely a TLS-ALPN challenge sent to a TLS implementation that doesn't know how to respond. It's the TLS version of a 404 error.
There were no redirects when the error first started. I deployed Let's Encrypt on the NGINX server, just to see if that would fix the error, and it didn't make a difference either way.
Looking through the release notes of FortiOS 7.2.4, I just noticed this under known issues:
Bug ID 864703 - ACME client fails to work with some CA servers. I'm wondering if that bug is somehow only affecting new certificate requests, but not renewals. I'll ask Fortinet TAC about it.
How do your deploy Let's Encrypt; as Let's Encrypt is https://letsencrypt.org/.
Are your trying to say you have deployed TLS with a Let's Encrypt issued Certificate on your nginx server?
And how are you deploying TLS with a Let's Encrypt issued Certificate on your fortigate?
Ah. That makes me think that it's possible that the public DNS record was changed, and is now pointing at a system that isn't the one running the client. And maybe if there's a split-horizon-type internal DNS that hasn't changed, the users actually running the system wouldn't have noticed?
Let's Encrypt. I have gotten new certificates from them using an older version of FortiOS, but apparently renewals must not be affected by this bug, because I've been on this firmware version for a while and never noticed any issues until I tried to request a new certificate today.
The FortiGate ACME client doesn't support external account binding (EAB), so the only one I was able to try is BuyPass. It failed with the following error:
None of offered challenge types for domain vpn.redacted.net are supported. The server offered 'http-01 dns-01' and available are: 'tls-alpn-01'.
So, this confirms that FortiOS has indeed switched the challenge method for new certificates to alpn-01 without telling anyone. Lovely.
So, for the record, this problem had nothing to do with a bad NGINX config.