How to use dns-01 auth on GCE by certbot-dns-google


#1

Hi,
I’m using Google Compute Engine and I now trying to get wildcard certificate for my domain by Google DNS.

I ran this command:
yum -y install certbot python2-certbot-dns-google --enablerepo=epel

When I try this command:
certbot certonly --dns-google -d hoge.com,*.hoge.com
Select: 2: Renew & replace the cert (limit ~5 per 7 days)
Output:
Renewing an existing certificate
Performing the following challenges:
dns-01 challenge for hoge.com
dns-01 challenge for hoge.com
URL being requested: GET https://www.googleapis.com/discovery/v1/apis/dns/v1/rest
URL being requested: GET https://www.googleapis.com/dns/v1/projects/single-mix-174909/managedZones?alt=json&dnsName=hoge.com.
Attempting refresh to obtain initial access_token
Encountered 403 Forbidden with reason “forbidden”
Cleaning up challenges
URL being requested: GET https://www.googleapis.com/discovery/v1/apis/dns/v1/rest
URL being requested: GET https://www.googleapis.com/dns/v1/projects/single-mix-174909/managedZones?alt=json&dnsName=hoge.com.
Attempting refresh to obtain initial access_token
Encountered 403 Forbidden with reason “forbidden”
Error finding zone. Skipping cleanup.
URL being requested: GET https://www.googleapis.com/discovery/v1/apis/dns/v1/rest
URL being requested: GET https://www.googleapis.com/dns/v1/projects/single-mix-174909/managedZones?alt=json&dnsName=hoge.com.
Attempting refresh to obtain initial access_token
Encountered 403 Forbidden with reason “forbidden”
Error finding zone. Skipping cleanup.
Encountered error finding managed zone: <HttpError 403 when requesting https://www.googleapis.com/dns/v1/projects/single-mix-174909/managedZones?alt=json&dnsName=hoge.com. returned “Request had insufficient authentication scopes.”>

The operating system my web server runs on is:CentOS Linux release 7.6.1810 (Core)

I’m not sure this plugin relates to gcloud command but I can get the domain list by gcloud command:
gcloud dns managed-zones list
NAME DNS_NAME DESCRIPTION
zone-hoge-com hoge.com.

Can anyone help?


#2

Whether or not you have a zone setup in your Google Cloud account, your domain is hosted elsewhere:

$ dig +noall +answer hoge.com ns
hoge.com.               14362   IN      NS      ns2.nktelco.net.
hoge.com.               14362   IN      NS      ns1.nktelco.net.

Are you planning to change the nameservers after setting this up or … ?

Edit: I notice your actual site is hosted on Google Cloud. What matters for wildcard certificates is where your DNS zone is hosted, not where your webserver is hosted. So using certbot-dns-google is not suitable unless you change your nameservers from nktelco to Google Cloud DNS.


#3

I’ve already changed my nameservers to google.

Thanks


#4

So far, the root nameservers still think your domain is using nktelco. But assuming you have updated it at your domain registrar (Network Solutions), it should eventually change.

Regarding your 403 error, could you show what permissions you have given to the service account assigned to your GCE instance? Certbot docs list the required permissions here: https://certbot-dns-google.readthedocs.io/en/stable/#credentials


#5

Sorry…
hoge.com is an example domain.
Actual domain’s nameserver is pointed to google dns for sure.

About Google API’s I think I set a right permissions.
Is there anyway to check the permissions writes by some commands?


#6

FYI
I’ve tested few gcloud commands.

Try add a A record:

$ gcloud dns record-sets transaction start -z=zone-hoge-com
$ gcloud dns record-sets transaction add --name test.hoge.com. --ttl 60 --type A --zone zone-hoge-com "192.168.1.1"
$ gcloud dns record-sets transaction execute -z=zone-hoge-com

And successfully finished transaction.

When I tried to add TXT record I got an error.
$ gcloud dns record-sets transaction add "Hello test" --name 4510life.com. --ttl 60 --type TXT --zone zone-4510life-com
ERROR: (gcloud.dns.record-sets.transaction.execute) HTTPError 409: The resource ‘entity.change.additions[1]’ named ‘hoge.com. (TXT)’ already exists

Their is already existing TXT record and I tried to add a new one.
But not sure gcloud command is able to add a multiple TXT records.

Does this error relates to the certbot command error??


#7

Not sure if this makes any difference…
But I would always use quotes around asterisks.
So, I would also try/test these just to be sure that is not part of the problem:
certbot certonly --dns-google -d "hoge.com,*.hoge.com" --dry-run
certbot certonly --dns-google -d hoge.com -d "*.hoge.com" --dry-run


#8

Thanks for your advise.
But it didn’t work also.
I found out I’ve missed the debug log file: /var/log/letsencrypt/letsencrypt.log
Maybe I have a lead for understanding the problem.
Here is the part of the log file.

2019-02-07 10:09:47,788:DEBUG:acme.client:Storing nonce: FybjjPZv0J3I8z1tQnT-Ev3nr-VYQ-_Z3uA7wimHYlU
2019-02-07 10:09:47,789:INFO:certbot.auth_handler:Performing the following challenges:
2019-02-07 10:09:47,789:INFO:certbot.auth_handler:dns-01 challenge for hoge.com
2019-02-07 10:09:47,803:INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/dns/v1/rest
2019-02-07 10:09:47,898:INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/dns/v1/projects/single-mix-174909/managedZones?alt=json&dnsName=hoge.com.
2019-02-07 10:09:47,898:INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
2019-02-07 10:09:49,067:WARNING:googleapiclient.http:Encountered 403 Forbidden with reason “forbidden”
2019-02-07 10:09:49,068:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
File “/usr/lib/python2.7/site-packages/certbot/auth_handler.py”, line 75, in handle_authorizations
resp = self._solve_challenges(aauthzrs)
File “/usr/lib/python2.7/site-packages/certbot/auth_handler.py”, line 132, in _solve_challenges
resp = self.auth.perform(all_achalls)
File “/usr/lib/python2.7/site-packages/certbot/plugins/dns_common.py”, line 57, in perform
self._perform(domain, validation_domain_name, validation)
File “/usr/lib/python2.7/site-packages/certbot_dns_google/dns_google.py”, line 70, in _perform
self._get_google_client().add_txt_record(domain, validation_name, validation, self.ttl)
File “/usr/lib/python2.7/site-packages/certbot_dns_google/dns_google.py”, line 113, in add_txt_record
zone_id = self._find_managed_zone_id(domain)
File “/usr/lib/python2.7/site-packages/certbot_dns_google/dns_google.py”, line 275, in _find_managed_zone_id
.format(e))
PluginError: Encountered error finding managed zone: <HttpError 403 when requesting https://www.googleapis.com/dns/v1/projects/single-mix-174909/managedZones?alt=json&dnsName=hoge.com. returned “Request had insufficient authentication scopes.”>

As you see, it says Encountered error finding managed zone so it might have a problem to find out the zone profile name which is: zone-hoge-com but how can I pass the zone profile name to certbot command?


#9

Looks like maybe the authentication is incorrect.
Have you reviewed @_az’s post above about the 403 error?


Dns-google dns-01 GCE Authentication problems