Certbot-dns-digitalocean blocked by cloudflare

slavaGanzin · December 8, 2022, 6:30pm

I ran this command: sudo certbot certonly --domains 'news-watcher.com' --preferred-challenges=dns --dns-digitalocean --dns-digitalocean-credentials ./certbot-do-credentials

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for news-watcher.com
Error finding domain using the DigitalOcean API: Read failed from DigitalOcean: Expecting value: line 1 column 1 (char 0)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

and letsencrypt.log has th2022-12-08 18:22:16,775:DEBUG:urllib3.connectionpool:https://api.digitalocean.com:443 "GET /v2/domains/?per_page=200 HTTP/1.1" 403 None
2022-12-08 18:22:16,777:DEBUG:certbot_dns_digitalocean._internal.dns_digitalocean:Error finding domain using the DigitalOcean API: Read failed from DigitalOcean: Expecting value: line 1 column 1 (char 0)is entry:

I've tried curl "https://api.digitalocean.com/v2/domains/" and get this cloud flare block page:
[lots of HTML]

api.digitalocean.com needs to review the security of your connection before proceeding.

[lots of HTML]

My web server is (include version): nginx/1.22.1

The operating system my web server runs on is (include version): arch rolling

My hosting provider, if applicable, is: digital ocean

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 2.1.0

So I'm blocked by cloudflare of using digital ocean. I've tried to launch chrome inside droplet and go through cloudflare checkup, it verified me, and chrome actually have access to api.digitalocean now, but it doesn't help with curl.

Any thoughts?

rg305 · December 8, 2022, 6:33pm

Hi @slavaGanzin, and welcome to the LE community forum

Does this file exist?:

If so, do those creds work?

Osiris · December 8, 2022, 6:45pm

@rg305 The problem isn't the credentials (yet): OP can't access the DO API due to the fact it's behind Cloudflare and Cloudflare is blocking connections from OPs droplet.

I don't have any idea beyond what OP already has tried. Probably needs to contact Cloudflare or perhaps wait. And of course find out the reason why Cloudflare is blocking the connection: maybe you're hammering the API?

slavaGanzin · December 9, 2022, 10:29am

I collect a lot of data parsing sites from that droplet. So maybe Cloudflare is blocking me because of that. But it's strange that DO API is protected from DO's own network.

slavaGanzin · December 9, 2022, 10:30am

Sometime ago I had the same issue. And I wait for couple of days and then blocking was lifted. Right now I don't have this luxury, because certificate lifetime will end in one day.

I'm not hammering DOs API for sure.

slavaGanzin · December 9, 2022, 10:31am

Yes. And the same command worked for me before

MikeMcQ · December 9, 2022, 2:56pm

Did this problem begin after upgrading to certbot 2.1 ?

Because a v2.2 was just issued to correct problems with plug-ins. I really don't know if the digitalocean one was affected (probably not) but thought I'd mention it.

When you try the failing curl could you add a -i value and show us the response headers. We can't duplicate your requests without your auth token but maybe these headers will provide a clue.

Again, just in case, the docs for the DO API (link here) show all the options and good curl examples

rg305 · December 9, 2022, 6:35pm

OR
[in the opposite direction...]
What version of certbot were you using during the last renewal?
[you could also try reverted back to that one]

slavaGanzin · December 9, 2022, 7:30pm

I downgraded to 1.29.0, nothing changed.

slavaGanzin · December 9, 2022, 7:31pm

curl "https://api.digitalocean.com/v2/domains/" -i

HTTP/2 403 
date: Fri, 09 Dec 2022 19:28:27 GMT
content-type: text/html; charset=UTF-8
cf-chl-bypass: 1
permissions-policy: accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()
referrer-policy: same-origin
x-frame-options: SAMEORIGIN
cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
expires: Thu, 01 Jan 1970 00:00:01 GMT
set-cookie: __cf_bm=6StxwR0pL7TaYB2GleNj0wMgjYWyAiTNbQvGWr26Uow-1670614107-0-AZmWmf5R4qNQ5K3sxsSqDIX6ougN1xIeee9uOqC2K7lz+D3HlMapsutqIOmUg/xfU1pWtT9v5gyweZiXoGqqzaoBGydrCNuptRos4b/b/PsH; path=/; expires=Fri, 09-Dec-22 19:58:27 GMT; domain=.digitalocean.com; HttpOnly; Secure; SameSite=None
server: cloudflare
cf-ray: 777024db2bda9195-FRA

<!DOCTYPE html>
<html lang="en-US">
<head>
    <title>Just a moment...</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=Edge">
    <meta name="robots" content="noindex,nofollow">
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <link href="/cdn-cgi/styles/challenges.css" rel="stylesheet">
    

</head>
<body class="no-js">
    <div class="main-wrapper" role="main">
    <div class="main-content">
        <h1 class="zone-name-title h1">
            <img class="heading-favicon" src="/favicon.ico"
                 onerror="this.onerror=null;this.parentNode.removeChild(this)">
            api.digitalocean.com
        </h1>
        <h2 class="h2" id="challenge-running">
            Checking if the site connection is secure
        </h2>
        <noscript>
            <div id="challenge-error-title">
                <div class="h2">
                    <span class="icon-wrapper">
                        <div class="heading-icon warning-icon"></div>
                    </span>
                    <span id="challenge-error-text">
                        Enable JavaScript and cookies to continue
                    </span>
                </div>
            </div>
        </noscript>
        <div id="trk_jschal_js" style="display:none;background-image:url('/cdn-cgi/images/trace/managed/nojs/transparent.gif?ray=777024db2bda9195')"></div>
        <div id="challenge-body-text" class="core-msg spacer">
            api.digitalocean.com needs to review the security of your connection before proceeding.
        </div>
        <form id="challenge-form" action="/v2/domains/?__cf_chl_f_tk=izyGHX6MIFQq9jKtNkDv9kismdDfiipwmGU68H6HJZ8-1670614107-0-gaNycGzNBv0" method="POST" enctype="application/x-www-form-urlencoded">
            <input type="hidden" name="md" value="_bJ2vj5Izb9MjfUCxGPhedWyTr06MSkt58vVR8gXn3w-1670614107-0-AQwvlaWouOacPQNgat0qJ77_PMYOLKGd6tU2Lzd8AtZQaXdU95KJUx3NqMvvIk33LyCABOT2aZIrXe31ch1OIFzKlTbwPtUqkf3AMgg2Mn4SDMraU1cvz7fqfNYJ_RMbLfiXjgyc6d7z5kyPpnX9rMZWFYiwvERP-W2icxszudv9tcrrmCMy4e-_WVnmWJSzGJ8fp-7t4iXmqdyKmDj6WmIfbUu-avLHypuylIplkhAAZWYJOlL-acpiWfB1YxObkZEJGBcVawX5LbxYkIMxTYBa75YOijgCiRAIUNFHYPhkB-5wDgYZLegD3zhKGhQ7g_igwYYdA2TighxlG_6QwG4EUMwwJGmGWSVP-PQC23kKGVhPrRXBN-B5r_0YE1QpwRAUvdIxZIYdeE9dlps1BjERg9Hi0XfKmy7GrpjZXc9gfBveXa2d_QMLGVKXFFyz_umPaEdBPPnjQwDfpqtWSNglMTMPdTR5hFiamZtBI7EmdWPfg_119vpRlE2HYy3koNWrjvjxg3JMT0f5mOraclW2Pf2sLxZJO7W0A-n7M3_q0jk4sKkDtCr2VYNcNF9yf4iNxCbUJYbS9t9pH69c4kPCVx_KW5RaHHBhB-Hm7JZ8VntuwHmbn2PQvkgjuFBsyA">
            <input type="hidden" name="r" value="ZXLtevf7cwgiCBYkloK39flXCgWrMUtr1ePeXg93gBY-1670614107-0-AQkm6Wcj8tbfYMV/N6ssP471HnxyYBFRdrito37AhK2jMEFmdLK2gcVPSaX/18DVm9vPKTMF6PM+O465Jfp7xsgkZxGFW4hQ4DL9SMA8Qkl6adxVAv7TrFdpq9m24gSuAVP7xRYlzV41Uyvx75xfiSugHW9WcmQP8JdeghmuEGhBjzf5h15ZRqYQU4njpkAcbgeQGr9oYjoLwNHlTqe+kD5Wr1A8Qqgy+ku8RSnNqcKVMhVaBJbL9YafEpF3V0k4miiG2sT0YeZuM2gpRwDEgItg2cjkrqi6bMZS4ZV6lp4vf8y4CaIqjq9VhBQC9wpCfm7fq1SAm385E24SlbQ8LVmZ/ZTKqiL0fn6oGsPOKfqcCEbSVKI+HGPxJndVOZu6XiTbp5fX5OvreBrsZCUFLojZ4OyeHM2RHxXfSWuAP3QZmsG62eGSv6GJid1sEdBCgocZilGtMZejPnWXRAfCxmki0Ef0moQ5NsCF/YNTiZjihuVW10vKyzMo+ZoYnyLYZe58YreMOtoThe5x/gqDJcL8+xidlw4CgX2trrZVtL/TQUFg3h5WV+wnElV32rrnymaILo5ulJidsXGdcl0jfaV82UfSA7acn0lka0y4OH0O/CK0Ye1iN2C9nFT3aPTga52qKVo9B3gu7GzzrG9MAajJlN9Owjf8y1ZbyMtUpXpiBT68fpYfNAHMWvnBLQWwrLISlJORGyIZURwPPsaUnev49dTf+w2rUIMWVvuJTHEvmSFZpzEavFTID1v8HSePbPWzGmO76gcb+qIpjYDaAGM4U5skIgDtvO9fY3E2TLlpK5foUEOvYcsGgjUeV0SwBGonq8rfAUYR/CZ/JNykADeVpcPQlpRhQL1wDU+R0i01JNR+q1aQ1Znql1Mt6RRk32GY4RENgNuMLb60q2O0RLvXrSfz9cox3v15NO2ZFaA7DKrKhP2M2skQq8FaxBrBTvPcuGW5yXTeVKwOcdNP5fbzlVXAXPRMc7H5FqecUo+6vTFSlIN/simXekk5GghBXc5TfKu6u2kTg3mfwVkMr4IVjThoOJqle8VuTXHc564kphO6yQkk0WoaNue438sRakULKP5M0vYTXRr0cGrS6v6vImVHDbrLUiVADcczuWZRhOup6wdddEPqpbSixL8Z3yr8u4SVYpcuXT/ppxEgNDJVI5oFmYYTICv+FZrsMN26emYBwkipClT+oanGf1Ovkug6xCOuL1E7fFraHZ5Jc2aB1rS6IjmDv5xL1k2SvYbW6hkuvxbV45thxcmEPCbxr3LWZ38EJXDkiND+/CysU7DS0MLh5fjsw/uLHcX2U9TsH453nKXYq1jdwd8LcFkqCJ9wVnxYLClUy+YSR6QFNSa7Q9EBAmvkyypFr4iqUFKdxyIlmWPMQg1gRENu43JTO7e09zLoDSYmtqYVUnRToCqlfBk3zSGqOOrPaJuP7NP0YBHOBna+ky9k/GnNX2zWFSSz4jyi+Jjqk+14rlW8yOy54WuAI73koy6iTb2LhDtctIO0KZgoxYEzhUpa6Nivqv5XXBJRB3igoVbk959S2koa8644SO102h4kqsqnMZlPOa0erB+aaHfsE/SKjtzPJAkbQ8xLGEyHYJtxOcFyQvIkkeJ/+ijQw7sGW4bgvmuz+gs5BWLG37T9fJHL3MeHkGWPqfDIKeXes0fDLimvCgn6bwml7wqX0Uvvi/ILtZ3MVL+THiE0vE118HgedRIcUw==">
        </form>
    </div>
</div>

rg305 · December 9, 2022, 7:45pm

Sounds like something they (and cloudflare) should hear about.

Name:      api.digitalocean.com
Addresses: 2606:4700::6810:b50f
           2606:4700::6810:b60f
           104.16.181.15
           104.16.182.15

MikeMcQ · December 9, 2022, 7:46pm

The 403 response is "Not Authorized". Along with the text from the html it looks like you need to consult with DO

Your response is different than if I do that request w/out any OAuth value so it looks like you formatted it right anyway

Me:

curl "https://api.digitalocean.com/v2/domains/" -i
HTTP/2 401
x-gateway: Edge-Gateway
x-response-from: Edge-Gateway
server: cloudflare

{"id": "Unauthorized", "message": "Unable to authenticate you" }

Osiris · December 9, 2022, 7:46pm

You're a minor version ahead 2.1 fixes issues some third party plugins have with deprecated and removed stuff in 2.0. But the DO plugin isn't third party and the fixes issue would otherwise have caused a severe error when loading the plugin, not something like this. (Which, IMO, is not Certbot related at all.)

rg305 · December 9, 2022, 7:54pm

Then we can definitely rule the ACME client out.

_az · December 9, 2022, 8:00pm

I think this is probably the answer. You'll have to talk to DO support.

slavaGanzin · December 9, 2022, 8:00pm

Guys thanks a lot, with your questions and guidance I understood that the problem is not with certbot/acme or even DO Api. It is about that shitty so-called protection by CloudFlare. And what is "great" about their protection? It's too easy to get around.

And now time for the real solution.

Get random free proxies from https://hidemy.name/en/proxy-list/#list (They never fail to deliver)
Use proxychains: sudo proxychains certbot [your argument list]
...
PROFIT

Thanks all for your time. I knew its' not a problem with certbot.

p.s. DO support is lazy and slow. So I opened an issue a while ago, still no answer.

system · January 8, 2023, 8:01pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.