The ACME service (directory) is unavailable

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: www.ichainverify.co.za

I ran this command:

It produced this output:

My web server is (include version): Windows IIS

The operating system my web server runs on is (include version): Windows Server 2016

My hosting provider, if applicable, is:

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): Yes, Im using CertifyWeb

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

The error message we get says ?

---- Beginning Request [DF.Demo.Web] ----
2022-10-12 19:01:04.273 +02:00 [INF] Certify/5.5.7.0 (Windows; Microsoft Windows NT 10.0.14393.0)
2022-10-12 19:01:04.992 +02:00 [INF] Beginning Certificate Request Process: DF.Demo.Web using ACME Provider:Certes
2022-10-12 19:01:04.992 +02:00 [INF] Requested identifiers to include on certificate: demo.assetsverify.co.za;api.demo.assetsverify.co.za
2022-10-12 19:01:04.992 +02:00 [INF] Beginning certificate order for requested domains
2022-10-12 19:01:26.334 +02:00 [ERR] The ACME service (directory) is unavailable.
2022-10-12 19:01:26.350 +02:00 [INF] The ACME service (directory) is unavailable.
2022-10-12 19:01:49.027 +02:00 [INF] The ACME service (directory) is unavailable.
2022-10-12 19:01:49.027 +02:00 [INF] The ACME service (directory) is unavailable.

Kindly indicate what could be happening, we have been using the same authority for 3 years, the issue started when we physically moved our servers however nothing changed apart from the public ip address ? Is it possible that our ip is blocked.

2 Likes

Welcome to the Let's Encrypt Community, Moshe! :slightly_smiling_face:

It appears that the machine from which you are running Certify the Web is unable to reach the Let's Encrypt production server, which implies a problem with your outbound internet access. Try visiting https://acme-v02.api.letsencrypt.org/directory from that same machine using a browser to see if you can reach it.

Tagging @webprofusion for awareness and support.

7 Likes

But unlikely Let's Debug results (click the link to view) is OK.

A while back Let's Encrypt started rejecting SHA-1 and TLS v1.0/v1.1 requests; this is just a grabbing at straw thought.

Also a few links on SHA-1 and TLS v1.0/1.1 with requesting certificates from Let's Encrypt.

  1. Rejecting SHA-1 CSRs and validation using TLS 1.0 / 1.1 URLs
  2. Email feedback: TLS 1.0/1.1 deprecation and SHA-1 deprecation
  3. Questions about TLS 1.0 / 1.1 deprecation for ACME requests

And this is what I get with curl.

$ curl -I http://www.ichainverify.co.za/.well-known/acme-challenge/letsdebug-test
HTTP/1.1 307 Moved Temporarily
Content-Length: 153
Content-Type: text/html; charset=UTF-8
Location: https://www.ichainverify.co.za
Server: Microsoft-IIS/10.0
X-Powered-By: ASP.NET
Date: Tue, 20 Dec 2022 16:06:36 GMT
2 Likes

What would it be unlikely? An IP being blocked by the API is for outbound request from their server. The Let's Debug test is for inbound to their server.

5 Likes

Using this online tool https://www.redirect-checker.org/ to show the redirects, I used http://www.ichainverify.co.za/.well-known/acme-challenge/letsdebug-test as the input and got this.

Result

http://www.ichainverify.co.za/.well-known/acme-challenge/letsdebug-test
307 Moved Temporarily
https://www.ichainverify.co.za
302 Found
/Account/Login
200 OK
Problems found:
You use a 302 redirect. This means, that the actually content is temporary not reachable and will come back soon.
To use a 302 redirection for generally moved pages is a bad idea. Search engine bot might not follow it or handle it as temporary.
For SEO this is also a bad idea, because no link juice will be transferred to the linked page.
You use a 307 redirect. A 307 is a temporary redirect in the HTTP protocol 1.1. Search Engine bots treat them as 302 redirect.
So it is a bad idea to use them, if your content has moved generally. Also link juice sill not flow to the linked page.

>>> http://www.ichainverify.co.za/.well-known/acme-challenge/letsdebug-test

> --------------------------------------------
> 307 Moved Temporarily
> --------------------------------------------
Status:	307 Moved Temporarily
Code:	307
Content-Type:	text/html; charset=UTF-8
Location:	https://www.ichainverify.co.za
Server:	Microsoft-IIS/10.0
X-Powered-By:	ASP.NET
Date:	Tue, 20 Dec 2022 16:09:51 GMT
Connection:	close
Content-Length:	153



>>> https://www.ichainverify.co.za

> --------------------------------------------
> 302 Found
> --------------------------------------------
Status:	302 Found
Code:	302
Cache-Control:	private
Content-Type:	text/html; charset=utf-8
Location:	/Account/Login
Server:	Microsoft-IIS/10.0
Set-Cookie:	ASP.NET_SessionId=fqh2gnu5knizk4qb2wdmbn1c; path=/; HttpOnly; SameSite=Lax
X-AspNetMvc-Version:	5.2
X-AspNet-Version:	4.0.30319
X-Powered-By:	ASP.NET
Date:	Tue, 20 Dec 2022 16:09:51 GMT
Connection:	close
Content-Length:	145



>>> /Account/Login

> --------------------------------------------
> 200 OK
> --------------------------------------------
Status:	200 OK
Code:	200
Cache-Control:	private
Content-Type:	text/html; charset=utf-8
Server:	Microsoft-IIS/10.0
X-AspNetMvc-Version:	5.2
X-Frame-Options:	SAMEORIGIN
X-AspNet-Version:	4.0.30319
Set-Cookie:	__RequestVerificationToken=LxO3dSw9QdrtVGWXr5LLQVguyH3tYWcWvphrsBoaLdJbxJVZop4-NkqomPBU1n7DvkbsoOyKqJlMcX_9ak3I_7AvQZr5J8NNl1Gb9uN6PDU1; path=/; HttpOnly
X-Powered-By:	ASP.NET
Date:	Tue, 20 Dec 2022 16:09:53 GMT
Connection:	close
Content-Length:	6393
2 Likes

This error message is uninformative, or at least, it doesn't provide any required detail what so ever. Please check with Certifys documentation on how to get a more detailed log and post it here :slight_smile:

5 Likes

That redirect would break the HTTP authentication request; As it removes the path and file being requested.

6 Likes

@mxsh your colleague Johan already has an open ticket with us to discuss and diagnose this issue but has not yet responded to our helpdesk reply (ref 2340).

You can confirm your server has connectivity to the Let's Encrypt API using PowerShell:
Invoke-WebRequest -Uri https://acme-v02.api.letsencrypt.org/directory which should return a StatusCode 200 (OK). Note that you must have TLS1.2 enabled to communicate with the Let's Encrypt API, I suggest using the IISCrypto tool by Nartec to configure their Best Practises mode to enable general purpose communication.

You are also not using the latest version of the app [which is 5.6.8].

Note that you can enable debug http logging by editing C:\ProgramData\certify\serviceconfig.json and set the "LogLevel" field to "debug" instead of "information", then restart the Certify background service and attempt your request again (just click "Request Certificate" on a managed certificate. The C:\ProgramData\certify\logs\session*.log file will then include the actual http conversation between you and the CA (which we are assuming is Let's Encrypt). [You can send this log file through to support at certifytheweb.com rather than posting anything here]

8 Likes

@lestaff it would be great if we could get a check on 169.239.70.148 to ensure it's not blocked. As the server has moved data center location it may be re-using the IP address previously occupied by something else.

3 Likes

Hi! Nope, we're not blocking that address

9 Likes

This is a neat trick that can help debug situations like this:

curl -v http://r3.o.lencr.org/

That request goes straight to the LE datacenter – so it is regulated by the ISRG Firewall Rules – but it's a simple HTTP request so it is immune to any client-side TLS issues.

If that HTTP request works, but the HTTPS requests for the API /directory fail, we can reasonably assume there is an issue with the client. If both HTTP and HTTPS requests fail, then there could be a firewall issue (either a LE firewall rule or client firewall rule) or a networking issue.

6 Likes

Hello @preston, welcome to the Let's Encrypt community. :slightly_smiling_face:
Nice to see a new Let's Encrypt staff.

6 Likes

Hi @mxsh
Moshe are you getting the help you are looking for?
If not what else would you like to know?
Do you need something explained in with an example?
Have you made any progress?

2 Likes

As far as I know, the OCSP responders are all behind a CDN for performance/caching. Akamai it seems to be. For example, the 2 IP addresses r3.o.lencr.org resolves to (88.221.25.162 and 88.221.25.176) are both Akamai IP addresses situated in the EU according to their WHOIS. And I'm pretty sure the LE datacenters are in the US.

4 Likes

I believe Cloudflare is used as the CDN. IIRC, Akamai is used for networking/routing into the data center.

See Request: Serve a debug text file via http behind ISRG's firewall - #8 by jcjones

5 Likes

Could be. But that would mean the router is done "under the water" from an Akamai EU data center to the actual LE data center in the US.

4 Likes

Regardless, it's a HTTP call that hits the same network rules as the HTTPS calls to the API.

4 Likes

Thanks this was resolved I believe it was our TLS version that was causing the issue, thanks for your help everyone.

4 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.