Cloud DNS providers with fastest nameserver propagation?

Hi,

I'm currently developing a cloud based (managed) acme-dns service for general purpose use. The design currently uses Cloudflare workers for the API and Google Cloud DNS for the DNS responses.

The main disadvantage currently is that the Google Cloud DNS TXT responses take a minute to propagate to the nameservers, so your acme-dns client (in my case Certify The Web) needs a delay for propagation which may not be ideal for other clients (even with a tiny TTL).

The original implementation of an acme-dns server serves the responses itself and doesn't require propagation. The alternative may be to implement a basic DNS server, but I'm trying to avoid that.

Does anyone have experience with other large scale cloud dns providers who have faster propagation between nameservers? I can stall the clients https request for 20 seconds or so via the API but not much longer than that (and that could have compatibility issues).

2 Likes

I think around a minute is fairly typical. I've been pretty happy with my personal use of AWS Route 53, but they also have an API you can poll to tell if all their servers have caught up with the change. Looking at the logs for my most recent requests, it looks like they took about 30 seconds, but that's also the default polling interval of the SDK so perhaps if I would have had it checked more often it might be even faster, at least sometimes? I seem to recall that sometimes in testing things would take longer than that first 30 second poll, though.

I think regardless, you shouldn't plan on DNS being synchronized within the span of a single HTTPS request, and need do some sort of polling or notification callback or the like.

And I don't have experience with any other DNS cloud providers, as AWS's has worked plenty well enough for what I've played around with. I kind of expect that all the major providers would be about the same in this area.

3 Likes

I've been able to regularly complete cert orders against Cloudflare's own DNS with sleep times at 15 seconds. I haven't tried to go any lower yet. Though if I immediately revoke the authorizations and try again with the same names, things fail unless I wait about a minute for some cache somewhere to expire. The TTLs on the records are set to "Auto", so this may just be the 1 min cache on the validation servers and I might be able to explicitly set them to something less than a minute.

4 Likes

I don't know the answer to your actual question, but I am interested in the security issues with this system. As far as I understand it, you're trying to develop a service where you act as the DNS server which would normally be run by the user self, in the case of acme-dns, right?

Would that mean you have total controle over the dns-01 challenge for the users using your service? Is this even allowed by the Let's Encrypt CPS et cetera?

4 Likes

I don't see how it's any different than me paying AWS to host my DNS, or any other hosted DNS provider. They have total control over answering any challenges for my name, if they weren't being honest about only me having control over my own zone.

4 Likes

@petercooperjr True. However, I've got a little bit more trust in the bigger companies compared to $random-person-on-the-internet :wink:

4 Likes

Good question for LE STAFF! (Did'nt tag um)

3 Likes

Sure. But in terms of it being "allowed", since whoever controls DNS for a name is who controls the name, that's what all the CAs (not just Let's Encrypt) can use for determining if they can issue a certificate, whether that DNS provider is a large cloud provider or $random-person-on-the-internet. DNS is certainly the weakest link in the chain (with DNSSEC solving some of the issues but not all, and regardless rolling out pretty slowly as best as I can tell). I suppose it's certainly good to keep in mind from a domain name owner's perspective, that one needs to be mindful of where one has DNS hosting because it's the lynchpin to everything. And of course not just who runs the current DNS servers, but who can edit the delegation from the TLDs for your names.

But regardless, I don't think Let's Encrypt's policies prevent people from hosting their DNS with $sketchy-hoster-on-a-street-corner-with-a-trenchcoat, and I don't really see how they could or should. (But sure, if you're planning on running a DNS server like @webprofusion is here, it's good to be aware of the security issues and trust that your users would be putting in you, and it sure doesn't hurt to make sure that message is clear.)

5 Likes

55zkl8

4 Likes

I sure do and today they're free... :wink: :wink:

It's not that I'm trying to plagiarize an entire service, it's that I'm trying to widen the market with additional advert... er... added-value products. What's your email address again? For security purposes only, of course.

3 Likes

Yes, security and trust is a challenge and that has to be acknowledged when providing this to users. I can't think of a single reason why I'd personally want someone else's certificate though, perhaps I'm not devious enough!

My motivation for this is that I have hundreds of thousands of users who often need a simple and least privileged way of handling DNS challenges. I did contemplate shipping a copy of acme-dns that was only spun up during challenges but the requirement for everyone to open port 53, delegate DNS and the amount of scary stuff on the internet randomly talking to it put me off. Plus you just know they'd never update it unless it stops working.

For many users running a webserver is hard enough - so configuring, running and maintaining an acme-dns server is just a step too far.

Yes, it does require trust though, it's a choice. I don't expect (for example) experienced Linux admins to use this service, I expect them to build acme-dns from source and run their own servers entirely built from source, after all, you can't trust someone else's binaries and they're not even digitally signed and their home git server was comprised anyway :slight_smile:

This is still an experiment and I'm not going to point people to it as a production service until I'm confident it's good enough. Once it's working and cleaned up I'll open source the various components but really it's much simpler than acme-dns itself.

Yes, I would indeed sell this service for a nominal fee so that I can then provide support for it. There is no way to run acme-dns (or any service) at zero cost because somebody somewhere pays for compute, data transfer and maintenance.

4 Likes

How about...
Hey I see that you have control for cert registrations for domain.X - how much would it cost me to get a cert from you for that domain?

You don't need to be devious to be bribed into doing something for someone who is.
[also anyone who "works for you" could also be at risk of such bribery as well]
["works for you" is just a nice way of saying "can admin your systems"]

3 Likes

Thanks, I can see the argument against it, it's convenience vs control. If you were throwing money around to nefariously obtain a cert the cheapest route is probably to bribe someone who works in sysadmin at the target company. I guess we could ask the acme-dns author if anyone has ever offered him money for the keys to https://auth.acme-dns.io/register or check with the various DNS providers to see which have ever been directly compromised to acquire a cert. Or ZeroSSL etc. I know for instance you don't really trust Cloudflare (as an example), so that's setting the bar pretty high, but I do understand your concerns.

I'm interested in solutions though, if you can think of a way to solve the problem with minimal complexity for end users then I'm all ears. Ultimately I was to help people get stuff working.

4 Likes

I don't know if this would actually be easier, but it might be possible for you to be a "reseller" or "partner" (not sure exactly what the right term is), where the users have an account directly with the cloud provider, and you're just involved in the initial setup and putting in whatever hooks are needed. So that their "contract" is with a large presumably-more-robust organization, rather than with you.

This is assuming of course, that the large organizations are actually going to be better for this sort of thing than a smaller player who does know what they're doing, which may or may not be the case. :slight_smile:

3 Likes

Just for some light, related reading...

2 Likes

Thanks Peter, I like the idea. Some Cloud vendors offer a marketplace for vendor solutions so there is scope to offer something through those (azure, aws etc) where the user just uses the service (as a set of functions/lambda etc) via their own cloud. They could of course just spin up acme-dns in docker but then we're back to issues of configuration and on-going management, likewise if they spin up their own cloud service instance there is still DNS delegation to configure (in order to use their own domains to host the final TXT records). Looping back though, for users that have that level of sophistication they will probably just use DNS validation via their existing cloud.

The subset of users who are not using a commonly automated DNS provider (or who can't) and who are not willing to run their own acme-dns service are probably the one's I'm targeting. It's typically small businesses without dedicated system administrators or otherwise relatively unsophisticated infrastructure.

It's an interesting problem, I've been mulling it over off and on for a couple of years. I see so many users still using manual DNS validation (the worst possible user experience and so difficult for users to get right) that I do think there is substantial room for improvement regarding DNS validation as a whole.

The lack of common DNS APIs is still a massive (elephant-in-the-room sized) issue for acme clients because there are literally thousands of smaller DNS providers and many only offer web UIs for updates (no API), meanwhile even those with APIs are potentially endangered if your ACME client has one line of bad DNS API update code.

4 Likes

I currently use CloudFlare, NS1, and Hurricane Electric for 2 domains. All of them are pretty fast. You'd see that your records have propagated almost instantly using dig. I haven't had a delay of 1 minute in record propagation while using Google Cloud DNS but sure enough, it was not an acceptable duration for acme verification.

Try Cloudflare if you are fine with more but basic features. Try NS1 if you want less, but more powerful features.

3 Likes

Due to some (typically large) providers using anycast for their nameservers, you can't really prove that all instances of that nameserver across the globe have been updated with a simple dig query. You'd need to be able to run the query from multiple vantage points. This is ultimately what Let's Encrypt does with multi-perspective validation.

5 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.