Heroku ACM stuck on "Dns Verified"

My domain is: webhooks.datocms.com

We are using Heroku with their ACM tool, so do we not run any command by ourselves.

On Sept 17th we had to move some domains from one Heroku app to another Heroku app (but within the same region) to circumvent an issue that Heroku had at that time on their infrastucture.

Later that day, when we moved back the domains into the original Heroku app, their ACM tool was no longer able to issue a certificate for the specified domain. It always remains blocked for some hours on "Dns verified" and then it get marked as "Failed".

This is very strange, since it's the only domain having problems with that application. The others 7 domains on that application got their certificate issued with no problems on Sept. 17th.

I think this topic is related, but I cannot reply on that thread anymore :frowning:

I've contacted their support too, but the problem persist.

Is there something that is blocking a certificate generation for the specified domain from your (Let's Encrypt) side? Maybe the domain went into a blocklist? Triggered a rate limit? Something in your logs that maybe could help Heroku engineers to fix this?

Thank you very much for any help you can give us

Based on the public certificate transparency records (as aggregated by crt.sh):

https://crt.sh/?q=webhooks.datocms.com

You've created 5 certificates for this domain name in the past week. (The interface is a little confusing, since it shows both the "precertificate" and final "leaf certificate" for each one, so it may look like 10 entries since 2021-09-15 but there are really only 5 certificates.) So I'm guessing you're hitting the "duplicate certificate" rate limit if you're trying to make another certificate. I don't know anything about Heroku, but you probably would need to work with them on why they can't use one of those existing 5 certificates that have been made and are trying to for some reason make another one. Perhaps they're creating them fine, but not managing to install them for some reason?

2 Likes

Slightly offtopic note: add &deduplicate=y to the URL to see only one of the types. Note that you can still see pre-certs as wel as real certs. But by deduplicating, you'd see just one of either for a pre-cert/real cert pair. (Rather randomly I believe..)

Also note that this feature is selectable from the UI behind the "advanced" button or something like that.

2 Likes

Thanks @petercooperjr :slight_smile:

I'll contact their support again. If we hitted that rate limit, then I think we could just try again next friday. Let's hope!

Thanks for the tip @Osiris !

1 Like

It was indeed a rate limit, because today I was able to generate a new certificate successfully :slight_smile:

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.