During secondary validation: No valid IP addresses found

Help
1

Since 3 weeks letsencrypt renewals have been failing on one of my servers due to Seconday Validation. I’ve setup a weekly cron for renewal (command below).

Referred this but it doesn’t seem to help.

I tried a manual renewal today using the same command and it succeeded. Unfortunately that over-wrote letsencrypt’s log so I can’t provide more details.

I’ve faced similar issues on some AWS instances also.

My domain is:
cms.dbf.ooo

I ran this command:
sudo service nginx stop && sudo /opt/certbot-auto renew && sudo service nginx start

It produced this output:
During secondary validation: No valid IP addresses found for cms.dbf.ooo

My web server is (include version):
nginx/1.14.0

The operating system my web server runs on is (include version):
Ubuntu 18.04

My hosting provider, if applicable, is:
Azure

I can login to a root shell on my machine (yes or no, or I don’t know):
yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 1.8.0

2

That’s strange. Does it happen every time you try?

sudo /opt/certbot-auto --pre-hook "service nginx stop" \
--post-hook "service nginx start" renew --dry-run
3

Hi @Manate

checking your domain the ooo zone is a little bit curious - https://check-your-website.server-daten.de/?q=cms.dbf.ooo

Three DS in the parent zone. And 5 DNSKEY:

5 DNSKEY RR found

Public Key with Algorithm 8, KeyTag 28386, Flags 256

Public Key with Algorithm 8, KeyTag 32704, Flags 256

Public Key with Algorithm 8, KeyTag 32919, Flags 257 (SEP = Secure Entry Point)

Public Key with Algorithm 8, KeyTag 41890, Flags 257 (SEP = Secure Entry Point)

Public Key with Algorithm 8, KeyTag 62653, Flags 256

Rechecked with dig, the message is large:

:~$ dig DNSKEY +dnssec ooo. @a.nic.ooo.

; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> DNSKEY +dnssec ooo. @a.nic.ooo.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40714
;; flags: qr aa rd; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; COOKIE: 15cf4aef3f1455a9bcf4ccca5f607752967065e220b36008 (good)
;; QUESTION SECTION:
;ooo. IN DNSKEY

;; ANSWER SECTION:
ooo. 3600 IN DNSKEY 256 3 8 AwEAAbHB7Akr3PbVvUnDgL03osiGeyl1lsBK2jSE6Aw14kSPyAQzLdzH aGlg4rjSaBT+8WFBa1VnFWCEPY9fkt8bLWYryBJZbMPpfEONCJMGxArc iTHXZJljWk02AM57bDlj2F035E/g6kZro0yH6cz+QKNSjR5vINFZQO4v 10T4xr7z
ooo. 3600 IN DNSKEY 256 3 8 AwEAAcLmqJaZuwVF8bBoxHgS3W+8zWTdT0hew5fm5jM35x0byxViEadx RpcCCi/w/6iUR2u5/iLPXRfY3YJFiqPecZWqzhGNGKj7+/QeyDGFuOfl gCVwjlUuv4fGl9T3E2+YURbWwf7qCt2Ep3VHHvZx2EWzldGjTclZu+u5 rAdyzcM5
ooo. 3600 IN DNSKEY 257 3 8 AQOr251LjWn/vdZF5gmNG7uCILJLtEEJqsaf6KcaYcI7XYTjKqBo35aq Cn6f/laV01AhnNWH3Ork42bmgRbgd3/tB3P1Mp9t5CUjmgNnr+38HSb6 RE/wLhAh2ry6gIj/xAh7e1kGLs/LxXLY37IjqcoWUYYr/jyEbXB33J8y C7iGefTGvttmu+7HFk0kpnZSBOOnwi2e2Oa8RUVcrL5xl3XlQZTJGaP8 MUgqE/Ymqu63MVJjguMqwp5XZcsXS8fWBY9v4feFnc85zEzF9WpNOqgc PwPdW+zrOT+G6RStx9lDaRbDYDcx2UKhd+uemCGr0S3IcEMAUZPkkfmh 7E6OIQTn
ooo. 3600 IN DNSKEY 256 3 8 AwEAAak7+uytRjUS/qiWLCtFWi021fL7+8jJirqSGJT1iAQRhsVfKFGi nqMg92RlDTrFzliRRBVfYU8xg09kjvscL1+T9g2fPX3pJZ9kYarT5pUL bk5uU2NY9oYDEVGQixIShgbil4ON1aaKIq0N/DHy+PeLz+cTV4GcrkkZ UX8eROab
ooo. 3600 IN DNSKEY 257 3 8 AwEAAaXoh87wSWLKDFy9Wd5M975GQ2QrIFKJsQX51zzeAyYRiRnR2Gs6 QD6dAjZItbaswvxIOprYaqsk77r2qsOhGdHSWg/ipMoIoJAIjck4PDUF onQuwwtqGpHJfmYtbp1mLg6QCrOnwUxVLIhY0BpaKxl11+U2kQC/n1rW 6t7QtbfkmvEWmTBCpH3gzf/wBOdAOleQkIaU8PC6mDo2eXROJIIjYdWu 5e3FFSn2TdQLljfuiOVB+drTLQ58Ydq7jou/MYknwg/LeqmNGNRYsnOu xnIaRvO0QOYw8ZIxsPXRlO19ZVBGOZTDz6OmPGMT33TWp/xQn/BXlagX ZVOrBUB0IMs=
ooo. 3600 IN RRSIG DNSKEY 8 1 3600 20200920003045 20200820173546 41890 ooo. MRzNNwB4M7XsSLhiv/bF4LcYZPoTPubY491/n9dv8oNwxJTVO498JruC O/nMk964tHKnZzy6sgf0D6Is8o/j+BpqSCOqg2enc7bdmzqxBkUC6Tzy gT+AGfNH8nXWObJy91EcvOh9P1QKiFr+4A8qtW91gDJIY1tx5q+uDqdi 8GN4uOmfS1dkcNhL+l6G+YgOw3uzovfM9fU/r6JohQKrrCbnURaESEEX uulSKsWD7sjMM0ERfNcWHrt+rjwm/KveZQS9gQch+IidmZhp8mucDqg4 P2l3nvMRT2+NwDmO2uKMTXdEfrjT6qIe/nFicsY/Xks+pl6qmxrn4jCk AG4xJw==

;; Query time: 51 msec
;; SERVER: 2001:67c:13cc::1:33#53(2001:67c:13cc::1:33)
;; WHEN: Tue Sep 15 10:12:02 CEST 2020
;; MSG SIZE rcvd: 1345

1345 Bytes.

But you can't change that. May be it's a key change, so it's a temporary problem.

These DNSKEY are checked because the ooo zone uses DNSSEC. So you can't skip that.

Sometimes such checks are too long -> that's the unspecific error message

No valid IP addresses found for cms.dbf.ooo

4

Nope. Like I said, I ran it manually today and it succeeded. I had been working fine a couple of months earlier, then resolved itself for once and then happening again.

5

Hello @JuergenAuer,

But you can’t change that. May be it’s a key change, so it’s a temporary problem.

Its been happening for more than a couple of months, so maybe that's not a temporary issue.

I'd admit that I'm a bit new to this arena, so many of the details are alien to me. What else should I check to narrow down or solve this issue?

Thanks,
Manate

6

How do you have your automatic renewal configured?

By default1, Certbot will try to renew 30 days before a certificate expires, twice a day.

That makes ~60 possible opportunities for your certificate renewal to succeed, despite any temporary issues that are out of your control, such as those which are the fault of the operators of the ooo TLD.

If @JuergenAuer is right about the cause, there’s not really anything you can do about it, apart from rely on scheduled renewals to succeed eventually.

1. Assuming you followed the advice about setting up automated renewal here: https://certbot.eff.org/lets-encrypt/pip-other

7

I have a weekly cron job setup which runs at midnight of Sundays to attempt renewal. Since its failing for about a couple of months now, I'm a bit skeptical that it will eventually succeed. I've scheduled it at midnight to avoid downtime during peak hours.

I think I have an option to run it at a different time. That will check if something is wrong with the time. Apart from that, anything else I can check for further diagnosis?

Thanks,
Manate

8

That's bad. Never run an update 00:00 / 01:00 etc.

Add a random number of minutes (maybe between 11 - 55 minutes).

9

Do not hesitate to run it once a day, that isn’t too frequent. Probably from /etc/cron.daily that even randomize the starting minute within the given hour.

10

Ok, I’ll try all the suggestions and post back.

Thanks for the support so far!
Manate

11

I have the same problem:
A cronjob 'checkAndPullCert.sh' checks at 3:00 o'clock for domains with certificates expire in less then 20 days and renew these.
A second cronjob 'ssl-cert-check.sh' cheks at 4:00 o'clock for domains with certificates expire in less then 17 days and send a email warning for these.
Since 3.9.2020 I have 3 email warnings. So the checkAndPullCert.sh-job tried 3 renews unseccessfull, e.g. this night:

I run then checkAndPullCert.sh manually in the morning like the cron-job and there is no error. Certificates a always renewed.

12

That’s interesting, thanks for posting. That gives weight to the theory that running your cronjob exactly on the hour increases the chance of hitting issues like this.

@mbudnick do you mind sharing what timezone 3:00 and 4:00 is in?

13

CEST +0200 :clock2: :earth_africa: :de:

14

Hi @mbudnick

Then change that. That's bad.

May be use 03:18:22 or something else if you can't add a random value.

But 03:00 is bad.

15

I have changed to 3:21 o’clock. This weekend reached the next domain the 20 day limit. So I will check on monday the log.

1 Like
16

The last renewal run successfully. The next is in 4 days.

17

The renewal run successfully again. Apparend changing the time was the key.

18

I am seeing exactly this error every few weeks and it usually goes aways after a day of repeated attempts (Apache/mod_md). Since Sunday, I see it for one of my domains and it has not unfixed itself since.

{"identifier":{"type":"dns","value":"eissing.org"},"status":"invalid","expires":"2020-10-05T22:00:18Z","challenges":[{"type":"http-01","status":"invalid","error":{"type":"urn:ietf:params:acme:error:dns","detail":"During secondary validation: No valid IP addresses found for eissing.org","status":400},"url":"https://acme-v02.api.letsencrypt.org/acme/chall-v3/7534046583/I1LhrA","token":"XXX","validationRecord":[{"url":"http://eissing.org/.well-known/acme-challenge/I5DWm3iKYXOUI3iC7nXF6S2CVVt4v7QTF3W30tfh2jc","hostname":"eissing.org","port":"80","addressesResolved":["217.91.35.233"],"addressUsed":"217.91.35.233"}]}]}

I'd say there is something fishy going on here. The DNS records for this domain have not changed for months, so it is not a missing dns update.

19

Hi @icing,

Do mod_md's renewal attempts on this deployment consistently fall on the hour?

I noticed the authz you posted was created at 0 minutes past the hour, and from a quick peek, it doesn't seem like there is any random factor for when next_run is calculated.

20

The attempts repeat hourly, but the first try is depending on when the server is restarted/-loaded. My errors happen for example at Wed Sep 30 10:39:22.839114 2020 and so on.

Where did you see that it falls on the hour?

I agree that some more randomness here would not hurt, but the underlying failure issue is not connected to the exact hour, as far as I can see.