During secondary validation: DNS problem: SERVFAIL looking up A for - the domain's nameservers may be malfunctioning

Hi,

I don't have access to the client's DNS but they changed the A record about two hours ago:
shop.agit-global.com 54.176.144.117

According to mxtoolbox.com and testing from my personal machine, the A record is set up correctly. However, when I run the command to issue the cert, I am running into trouble. I am worried about hitting my 50 weekly attempts.

I have used the command to order certificates for other domains dozens of times and have never had any issues. Usually the command works at first attempt within minutes of the DNS A record being configured. Thanks for the help!!

My domain is:

I ran this command:

cd /usr/local/letsencrypt
./letsencrypt-auto --apache -d shop.agit-global.com

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for shop.agit-global.com
Waiting for verification...
Challenge failed for domain shop.agit-global.com
http-01 challenge for shop.agit-global.com
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: shop.agit-global.com
   Type:   dns
   Detail: During secondary validation: DNS problem: SERVFAIL looking
   up A for shop.agit-global.com - the domain's nameservers may be
   malfunctioning

My web server is (include version):
AWS EC2

The operating system my web server runs on is (include version):
Ubuntu 18.04.5 LTS

My hosting provider, if applicable, is:
AWS

I can login to a root shell on my machine (yes or no, or I don't know):
yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 1.8.0

2 Likes

Welcome to the Let's Encrypt Community :slightly_smiling_face:

This has been a very common issue lately. I believe it is based on the secondary validation servers being overwhelmed at midnight. Please try changing the time the process runs to a somewhat random time several hours and minutes earlier or later.

@lestaff

Another secondary validation failure

1 Like

That doesn't seem quite right...

Please show the outputs of:
/usr/local/letsencrypt/letsencrypt-auto --version
which certbot

2 Likes

Thanks for your help with this! I ended up copying the existing cert over from the old server the site was hosted on which buys me some time to get this sorted.

Meanwhile, I am still unable to request a certificate from Let’s Encrypt due to the same issue. I am by no means a DNS expert and I am wondering if there might be something about this domain’s nameserver which LE doesn’t like.

Here are the commands for @rg305

/usr/local/letsencrypt/letsencrypt-auto --version
certbot 1.8.0

which certbot
/usr/bin/certbot
2 Likes

What says?:
/usr/bin/certbot --version

1 Like

Also:

2 Likes

@rg305

Thanks so much for helping me figure this out! I really appreciate it.

/usr/bin/certbot --version
certbot 0.27.0

I will pass the DNS Lookup Debug on to the client whose IT department manages their DNS. Is there any additional detail I can provide them with that would explain what they did wrong?

Thanks!

2 Likes

Step #1: uninstall that version
sudo apt remove certbot

2 Likes

hmm...
For a site name that includes the word "global", their DNS is quite lacking from that perspective.
I don't know if "wrong" is the best adjective in this case.
But it could surely have been designed/made "better" / more resilient (more "global").
Have a look at: https://dnsspy.io/scan/agit-global.com

2 Likes

Furthermore, gTLD DNS servers only show three DNS servers for their domain:

agit-global.com nameserver = ns01.idc.hinet.net
agit-global.com nameserver = ns02.idc.hinet.net
agit-global.com nameserver = ns03.idc.hinet.net

While other public DNS systems show six:

agit-global.com nameserver = ns.hinetidc.net
agit-global.com nameserver = ns1.hinetidc.net
agit-global.com nameserver = ns2.hinetidc.net
agit-global.com nameserver = ns01.idc.hinet.net
agit-global.com nameserver = ns02.idc.hinet.net
agit-global.com nameserver = ns03.idc.hinet.net

[some kind of synchronization/configuration issue there...]

LE will follow the gTLD DNS and thus is limited to only the first three.

2 Likes

I will pass this info on to the client who manages their DNS. Unfortunately this likely means we will have to get a certificate from a different provider. Which is so strange to me because the certificates would issue just fine from LE on the old server. Thanks again for all your effort

2 Likes

Any other provider will likely face the same issue.

2 Likes

Hi @snuckyr

checking your domain via https://check-your-website.server-daten.de/?q=shop.agit-global.com

  • Your name servers are ok, different ip addresses answer, no critical timeout
  • But: You don't have a CAA entry

13. CAA - Entries

Domainname flag Name Value ∑ Queries ∑ Timeout
shop.agit-global.com 0 no CAA entry found 1 0
agit-global.com 0 no CAA entry found 1 0
com 0 no CAA entry found 1 0

Sometimes (only sometimes) it helps to create a CAA with the longest domain name shop.agit-global.com.

Some name servers have problems with a missing RR. But if there is a RR (your CAA entry), all works.

The error message from letsdebug.net @rg305 has shared ( During secondary validation: DNS problem: SERVFAIL looking up A for - the domain's nameservers may be malfunctioning ) says: It's a problem with the CAA check.

Curious: Running the same check with my local unbound (letsdebug and Letsencrypt are using unbound too), there is no Servfail visible.

Same with unboundtest - https://unboundtest.com/m/CAA/shop.agit-global.com/PYCXSWTX

NoError. Different results between unboundtest and letsdebug are rare.

2 Likes

The original issue is likely the same secondary validation failures that have been happening all over the place. I mentioned this in my very first post. I have noticed that Let's Debug has been returning strange things quite often lately. My results for dig and Let's Debug have conflicted way too often. Glad your results confirm that @JuergenAuer. I might email the Let's Debug guys just to check. Regional issue?

2 Likes

Pay no attention to the 3 DNS servers versus the 6 DNS servers...
[now re-read that with an emphasized sarcastic voice in mind]

1 Like

That may very well be an issue. I just wanted to bring attention to:

  • The original error is most definitely a secondary validation failure
  • Let's Debug has been returning some wonky results lately (lots of failed CAA and A lookups)
1 Like

It's time to reboot the entire Internet again... Get Al Gore on the line!

1 Like

He's floating in orbit from all his hot air!

2 Likes

I agree with @griffin.

I don't use letsencrpyt but they are used by a web hosting company that we use. I see the same CAA (when no CAA exists) failures along with A/AAAA failures when in fact the A/AAAA records are there and resolving correctly from everywhere except the letsdebug tool. I can run that tool against different zones hosted on the same DNS that return OK but other zones on the same DNS using the same name servers fail all the checks. Its maddening to say the least that companies rely on your service to generate certificates but have no idea what the problem is when it doesn't work.

So why I am here...can anyone tell me what letsdebug.net is actually doing or looking for? Is it as simple as querying the internet or the NS for any given domain for CAA/A/AAAA records? It would be great if there was an output or detail that gave information and logs as to why it failed.

1 Like

Thanks for joining and providing valuable intel. I have added your report to our internal topic tracking Let's Debug trouble sightings. I would like your post, but I'm out of likes right now.

@_az

As the developer of Let's Debug, you are definitely the one to field @thisisbroken's questions.

2 Likes