Challenge.request_validation returning status as invalid in DNS01 challenge type

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: check.lambdasam.tk, lambdasam.tk

I ran this command:
I wrote a ruby code to generate certificates for any sub-domain and domain name I provide.
I am using the recommended unixchales/acme-client for ruby.
I use cloudflare api to update DNS records for my domain. I could verify that txt records with challenges corresponding to subdomains required are updated successfully.
But request_validation is returning status invalid.
I tried for subdomain named “check” just now giving the same error.
check.lambdasam.tk
dig _acme-challenge.check.lambdasam.tk is showing challenge text properly.
As far as I could understand if a DNS entry shows up correctly as required by the challenge the request_validation should return status valid.
This whole setup used to work flawlessly till 1 months ago. I haven’t made any changes to my code since. I don’t understand what changed which broke all this. Please help.
I would like to mention that I could generate cert for this domain using other client as certbot.

Here is my code snippet
API calls generateCertificate method.

require 'cloudflare'
require 'openssl'

class HelperMethods
	def self.generatePKey
		private_key = OpenSSL::PKey::RSA.new(4096)
		# store it in a file with any encryption
		encryptPKey(private_key)
	end

	def self.encryptPKey(private_key)
		cipher = OpenSSL::Cipher.new 'AES-128-CBC'
		pass_phrase = 'my secure pass phrase goes here'

		key_secure = private_key.export cipher, pass_phrase

		open 'private.secure.pem', 'w' do |io|
  			io.write key_secure
		end
	end

	def self.decryptPKey
		if !File.exist?('private.secure.pem')
			generatePKey
		end
		key_pem = File.read 'private.secure.pem'
		pass_phrase = 'my secure pass phrase goes here'
		key = OpenSSL::PKey::RSA.new key_pem, pass_phrase
		return key
	end
	
	def self.setupClient
		private_key = decryptPKey
		client = Acme::Client.new(private_key: private_key, directory: 'https://acme-staging-v02.api.letsencrypt.org/directory')
		# mail id to get certificate expiry alert etc.
		account = client.new_account(contact: 'mailto:myemail@gmail.com', terms_of_service_agreed: true)
		# return kid
		key_id = account.kid
		open 'key_id', 'w' do |io|
			io.write key_id
	  	end
	end

	def self.initiateGeneration
		private_key = decryptPKey
		if !File.exist?('key_id')
			setupClient
		end

		@kid = File.read 'key_id'
		@client = Acme::Client.new(private_key: private_key, directory: 'https://acme-staging-v02.api.letsencrypt.org/directory', kid: @kid)
		@order = @client.new_order(identifiers: [@subdomain_name + "." + @domain_name])
		@authorization = @order.authorizations.first
		@dns_challenge = @authorization.dns
	end

	def self.initiateChallenge
		@challenge_name = @dns_challenge.record_name # => '_acme-challenge'
		@challenge_record_type = @dns_challenge.record_type # => 'TXT'
		@challenge_key = @dns_challenge.record_content # => 'HRV3PS5sRDyV-ous4HJk4z24s5JjmUTjcCaUjFt28-8'
		puts(@challenge_key)
	end

	def self.addDNSRecord
		# challenge name for dns-01 verification method
		@challenge_name = @dns_challenge.record_name # => '_acme-challenge'
		# challenge key for verification
		@challenge_key = @dns_challenge.record_content # => 'HRV3PS5sRDyV-ous4HJk4z24s5JjmUTjcCaUjFt28-8'
		# record type for verification
		@challenge_record_type = @dns_challenge.record_type # => 'TXT'

		# cloudflare credentials
		#registered email with cloudflare
		@email = 'myemail@gmail.com'
		# global api key
		@key = 'xyz123456789'

		Cloudflare.connect(key: @key, email: @email) do |connection|
			# Add a DNS record. We need to add TXT DNS record with auto-generated value 
			#to be verify domain ownership with Let's Encrypt
			zone_to_update = "#{@challenge_name}.#{@subdomain_name}"
			zone = connection.zones.find_by_name(@domain_name)
			zone.dns_records.create(@challenge_record_type, zone_to_update, @challenge_key)
		end
	end

	def self.verifyDNSEntry
		#dig -t txt @challenge_name.@sudomain.@domain
		#if found valid value then return true else wait
		cmd = "dig -t txt #{@challenge_name}.#{@subdomain_name}.#{@domain_name} +short"
		value = `#{cmd}`
		while value == ""
			sleep(2)
			value = `#{cmd}`
		end
		puts("challenge verified")
	end

	def self.completeChallenge
		@dns_challenge.request_validation
		while @dns_challenge.status == 'pending'
			sleep(2)
			@dns_challenge.reload
			puts(@dns_challenge.status)
		end
		@dns_challenge.status # => 'valid'
		puts(@dns_challenge.status)
		if @dns_challenge.status == 'invalid'
			puts(@dns_challenge.status)
			#cleanupDNSEntry
		end
	end

	def self.downloadCertificate
		#generate a different private key
		a_different_private_key = OpenSSL::PKey::RSA.new(4096)
		common_name = "#{@subdomain_name}.#{@domain_name}"
		csr = Acme::Client::CertificateRequest.new(private_key: a_different_private_key, subject: { common_name: common_name })
		@order.finalize(csr: csr)
		while @order.status == 'processing'
  			sleep(1)
  			@dns_challenge.reload
		end
		cert = @order.certificate # => PEM-formatted certificate
		file_name = @subdomain_name+"_"+Time.now().to_s+"_cert.pem"
		open file_name, 'w' do |io|
			io.write cert
		end	  
	end

	def self.cleanupDNSEntry
		@record = "#{@challenge_name}.#{@subdomain_name}.#{@domain_name}"
		Cloudflare.connect(key: @key, email: @email) do |connection|
			# Remove DNS entry
		
			zone = connection.zones.find_by_name(@domain)
			zone.dns_records.find_by_name(@record).delete
		end
		return "deleted succesfully"
	end


	def self.revokeCertificate
		private_key = decryptPKey
		cert_key = File.read 'cert.pem'
		client = Acme::Client.new(private_key: private_key, directory: 'https://acme-staging-v02.api.letsencrypt.org/directory')
		client.revoke(certificate: cert_key)
	end

	def self.renewCertificate
		# renew is just placing a new order and replacing the old certificate
		generateCertificate
	end

	def self.generateCertificate(params)
		#find if account private key file exist true then continue else generate new file
		@subdomain_name = params[:subdomain_name]
		@domain_name = params[:domain]

		initiateGeneration
		initiateChallenge
		addDNSRecord
		verifyDNSEntry
		completeChallenge
		downloadCertificate
		cleanupDNSEntry
		# certificateDispatch

		return true
	end
end

It produced this output:

My web server is (include version): Not Required

The operating system my web server runs on is (include version): Not Required

My hosting provider, if applicable, is: Not required. For DNS Cloudflare.

I can login to a root shell on my machine (yes or no, or I don’t know): NO

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): NO

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): unixcharles/acme-client 2.0.5 for ruby

1 Like

Hey,

Could you please print out the order URL in initiateGeneration, and then post it here on the forum?

puts @order.url

.tk nameservers have been really crappy and unreliable for the past 4-5 months or so. It’s possible that Let’s Encrypt failed on e.g. the CAA check for tk itself. That’s my main suspicion, anyway.

With the order URL, we would be able to understand what the reason for the invalid status is.

2 Likes

The IRONY here is trying to get a cert to secure a .TK domain!

Hello,
Here is my order url
https://acme-staging-v02.api.letsencrypt.org/acme/order/11648496/63171430

Also I would like to mention that I could successfully generate certificates for the same domain using certbot using the same DNS01 method.

1 Like

Thanks. My earlier suspicion was correct.

If you navigate to the authorization URL in that order, https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/23395009 , we see the true cause of the failure:

DNS problem: SERVFAIL looking up CAA for tk

:man_shrugging:

Unfortunately it’s just random chance. tk's nameservers are just unreliable.

There is a workaround, and that’s to create your own CAA record under lambdasam.tk. For example:

lambdasam.tk. 600 IN CAA 0 issue "letsencrypt.org"

This prevents the tk CAA lookup from happening, and avoids the failure.

3 Likes

Thanks a lot. This worked.
Sorry for using .tk domain. This is my test project and buying a premium domain just for testing wasn’t viable. I also stumbled upon this CAA issue with .tk domain while searching for probable cause but as certbot was working fine for me I thought it was some mistake at my end.
Finally thanks a lot for the workaround. You guys are doing great work.

2 Likes

If you already have ANY (better) domain, you should be able to use a single subdomain of it for testing.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.