Failure creating certificate acme challenge 404 error in BIG-IP F5 WAF

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: yatinuwara.ds.gov.lk

I ran this command: dehydrated script GitHub - dehydrated-io/dehydrated: letsencrypt/acme client implemented as a shell-script – just add water

It produced this output:
Processing yatinuwara.ds.gov.lk

I'm going to ignore the multiple times you pressed Ctrl-C during that dehydrated run for a minute..

Are you following some kind of guide by any chance? If so, would you care to share it with us?

Also, I'm noticing your BigIP appliance is answering with a very specific way to any ACME challenge:

osiris@erazer ~ $ curl -Lv http://yatinuwara.ds.gov.lk/.well-known/acme-challenge/test
*   Trying 43.224.124.166:80...
* Connected to yatinuwara.ds.gov.lk (43.224.124.166) port 80 (#0)
> GET /.well-known/acme-challenge/test HTTP/1.1
> Host: yatinuwara.ds.gov.lk
> User-Agent: curl/7.74.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
* HTTP 1.0, assume close after body
< HTTP/1.0 404 Not Found
< Server: BigIP
* HTTP/1.0 connection set to keep alive!
< Connection: Keep-Alive
< Content-Length: 35
< 
* Connection #0 to host yatinuwara.ds.gov.lk left intact
Challenge-response token not found.osiris@erazer ~ $ 

Notice the "Challenge-response token not found." response of your BigIP webserver? If I do a Google search on that sentence, I'm getting a hit for GitHub - fanceg/letsencrypt-bigip which seems to be something which integrates Let's Encrypt with BigIP. Are you using that piece of software too?

It might be helpfull if you'd tell us a little bit more about your situation, what you've already tried, what didn't work, the whole deal. Not just a single command with an output which contains three Ctrl-C's...

2 Likes

Hi Osiris,
Thanks for your response

We have more than 600 government websites behind the BIG-IP system. We have done almost 60% of certificates created and offloaded. Suddenly we couldn't create any certificate and got the below error. This error not only for one website. Now we can't renew or create a new certificate.
We use exactly the same script that you shared to integrates Let's Encrypt with BigIP ( GitHub - fanceg/letsencrypt-bigip).

[vsisadmin@host-192-168-170-21:Active:Changes Pending] ~ # cd /etc/dehydrated
[vsisadmin@host-192-168-170-21:Active:Changes Pending] dehydrated # ls -l
total 122
drwxr-xr-x. 3 root webusers 1024 May 9 10:46 accounts
drwxr-xr-x. 48 root webusers 3072 May 9 20:37 certs
drwxr-xr-x. 2 root webusers 1024 May 9 01:15 chains
-rwxr-xr-x. 1 root webusers 212 Feb 10 13:09 client-ssl
-rw-r--r--. 1 root webusers 193 Feb 10 14:19 client_ssl_profile.sh
-rw-r--r--. 1 root webusers 189 Feb 10 13:20 client-ssl-svr-name
-rw-r--r--. 1 root webusers 317 Jan 27 00:09 client-ssl-vs-acctach
-rwxr-xr-x. 1 root webusers 188 Feb 10 14:24 client-ssl-vs-acctached
-rwxrwxrwx. 1 root webusers 3681 Jan 27 00:16 config
-rwxrwxrwx. 1 root webusers 83678 Apr 23 05:33 dehydrated
-rwxrwxrwx. 1 root webusers 578 May 9 20:27 domains.txt
-rwxrwxrwx. 1 root webusers 4718 Jan 24 03:55 hook.sh
-rw-r--r--. 1 root webusers 620 Mar 8 2019 install.sh
-rw-r--r--. 1 root webusers 2078 Mar 8 2019 send_mail
-rw-r--r--. 1 root webusers 872 Mar 8 2019 wrapper.sh
[vsisadmin@host-192-168-170-21:Active:Changes Pending] dehydrated #

[vsisadmin@host-192-168-170-21:Active:Changes Pending] dehydrated # ./dehydrated -c -g

INFO: Using main config file /etc/dehydrated/config

Processing verugal.ds.gov.lk

We did the integration after got this issue, but didn't work!

Hi, Any updates on this issue!

Can anyone help me out with this issue? Are there any process changes or updates in letsencrypt site? Due to this lots of government websites affected!

Hi @Dineshan

checked GitHub - fanceg/letsencrypt-bigip

on 8 Mar 2019

So that tool used expired things, ACME-v1 and other. V1 is deprecated, you can't create new certificates.

So I don't think that can work. Update to a newer client.

1 Like

Hi @JuergenAuer

We used acme-staging-v02 for CA. This issue happens with that!
config.txt (3.5 KB)
dehydrated.txt (62.3 KB)

Path to certificate authority (default: https://acme-v01.api.letsencrypt.org/directory)

CA="https://acme-staging-v02.api.letsencrypt.org/directory"

Setting the CA server to the v02 endpoint doesn't automatically mean it's going to work. If your 0.6.0 version of dehydrated doesn't support the ACMEv2 (i.e., the RFC 8555 compatible ACME endpoint of Let's Encrypt), it's still not going to work.

That said, in your logs above I'm not really seeing any ACMEv1 vs. ACMEv2 problems. It returns invalid responses when asked for the http-01 challenge token. But I'm not experienced with dehydrated nor the BigIP script mentioned to say why.

1 Like

@Dineshan : Why do you use 0.6?

GitHub - dehydrated-io/dehydrated: letsencrypt/acme client implemented as a shell-script – just add water shows a 0.7 - 2020-12-10, may be GET as POST is the difference.

So you may use a dehydrated that can't work.

2 Likes

I think that would result in errors way earlier in the ACME process. I.e., not even getting an authorization to validate in the first place, while here we see authorizations being processed.

1 Like

Hi @JuergenAuer @Osiris

   0.7.1 script also same error!

Please read your error message:

There is an invalid result:

There

is something that looks validated.

Why? Please explain that difference and find the reason.

PS:

If a part works and another part not, it's a local problem you have to fix. And you (only you) know the difference.

1 Like

We couldn't figure out the issue from our side. Are there any findings?

Any updates or findings regarding this issue?

Unfortunately, I don't think anyone here is familiar with the F5 WAF or the script that you're using. Can you potentially get in touch with Gavin Fance, the developer of that script?

This forum is an appropriate place to ask for help with any Let's Encrypt integration, but our forum members sometimes don't know much about particular platforms or environments, and I think that's the case for your setup.

You might also want to try F5's DevCentral which is entirely populated by F5 developers and users. (Although, conversely, they may not all know that much about Let's Encrypt.)

https://devcentral.f5.com/s/