Using --expand to include subdomain - 404 not found

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: hub.onlineorders.org.uk

I ran this command: --expand (on certificate for onlineorders.org.uk)

It produced this output:Domain: hub.onlineorders.org.uk
Type: unauthorized
Detail: Invalid response from
http://hub.onlineorders.org.uk/.well-known/acme-challenge/oBz2yItwKxOEN6Gk54P4Njb01hP5KprTrl9fEfnBwUA
[90.251.249.231]: "\n\n404 Not
Found\n\n

Not Found

\n<p"

My web server is (include version): Apache 2.4.38 (Raspian)

The operating system my web server runs on is (include version): Debian10 Buster

My hosting provider, if applicable, is: n/a

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 0.31.0

Hi. This problem is really difficult. I have successfully secured another subdomain, admin.onlineorders.org.uk including manually editing the Virtual Host entries (/etc/apache2/sites-available/000-default.conf) and the (/etc/apache2/sites-available/000-default-le-ssl.conf), ports 80 and 443. I have checked the file structure in var/www/html and I have checked the IP address with my registrar. Identical entries work for one subdomain, but not the other. I am baffled. Could anybody offer any suggestions please?

Many thanks.

2 Likes

Welcome to the Let's Encrypt Community, Peter :slightly_smiling_face:

I've taken a look.

Please post the entire certbot command you used:

  • --expand is usually not the best route
  • an incorrect webroot is suspect here

You didn't include hub.onlineorders.org.uk on the certificate that it is currently serving, which is why it's currently insecure. I assume this is why you're trying to expand the certificate. This won't affect your ability to acquire a certificate though.

Certificate history

Notes

http://hub.onlineorders.org.uk/.well-known/acme-challenge/test

Not Found

The requested URL was not found on this server.

Apache/2.4.38 (Raspbian) Server at hub.onlineorders.org.uk Port 80

1 Like

Hi Griffin.

Thank you for your reply. I should mention that I tried to obtain a certificate with hub.onlineorders.org.uk as a stand-alone domain as well as part of the expand request. Same error both times.

Basically, I was only getting the error on the hub. prefix, so I decided to omit it from the certificate to see if I could secure admin.onlineorders.org.uk and it worked.

When you say webroot, I'm not sure exactly what you mean tbh. Could you explain?

Many thanks.

2 Likes

I have to run for awhile, but the webroot is the directory that is the root of your website. It's usually where your index file or homepage is located. That's why we need to know the full command you used.

Hopefully someone will pick up this topic before I return to continue helping you out.

1 Like

I would start with:
sudo apachectl -S

and go from there...

3 Likes

Hi. Thanks for the reply. I did as you suggested and these are the relevant parts which look ok to me:

*:80 is a NameVirtualHost
default server hub.onlineorders.org.uk (/etc/apache2/sites-enabled/000-default.conf:1)
port 80 namevhost hub.onlineorders.org.uk (/etc/apache2/sites-enabled/000-default.conf:1)
alias www.hub.onlineorders.org.uk
port 80 namevhost admin.onlineorders.org.uk (/etc/apache2/sites-enabled/000-default.conf:10)
alias www.admin.onlineorders.org.uk

and

Main DocumentRoot: "/var/www/html"

I have also specified a ServerName parameter.

Also, if you use a browser and navigate to http://hub.onlineorders.org.uk, you will get there just fine. (Note it's a mobile based site, so the desktop version will give you a blank page at present.)

3 Likes

Thanks for your reply yesterday. I have always used DocumentRoot but now I understand you perfectly. However, it's fine. It was my first thought too.

It's the standard /var/www/html with sites held in individual sub directories and referenced by the DocumentRoot entry in the virtual host configuration.

Regarding the exact command: sudo certbot --apache -d onlineorders.org.uk,www.onlineorders.org.uk,hub.onlineorders.org.uk,www.hub.onlineorders.org.uk,
admin.onlineorders.org.uk,www.admin.onlineorders.org.uk --expand

is the command that failed copied from terminal. Also, the same error applies if I try to get a stand alone certificate.

3 Likes

This is to be expected. A standalone certificate goes through the same verification processes as a combined certificate on a per-domain basis (with the exception of including a wildcard (*.) if the ACME client is implemented to force a universal challenge type).


That's a good confirmation for ensuring the success of the combined certificate later.


You really want to be using --dry-run for testing, which uses the staging servers and produces fake certificates that aren't saved, to prevent getting rate-limited.


As a start to solving this, I would like to suggest an improved command workflow. It uses --cert-name, which allows you to provide a name for your certificate and is far more powerful than --expand. It also uses -w to specify the webroot directory for any upcoming -d parameters. Please be sure the webroot directories are accurate. I assumed that the pairs of non-www and www share the same webroot directories.

Testing

sudo certbot certonly \
--cert-name onlineorders \
-a webroot \
-w /var/www/html/onlineorders.org.uk \
-d "onlineorders.org.uk,www.onlineorders.org.uk" \
-w /var/www/html/hub.onlineorders.org.uk \
-d "hub.onlineorders.org.uk,www.hub.onlineorders.org.uk" \
-w /var/www/html/admin.onlineorders.org.uk \
-d "admin.onlineorders.org.uk,www.admin.onlineorders.org.uk" \
--dry-run

Live

sudo certbot run \
--cert-name onlineorders \
-a webroot \
-w /var/www/html/onlineorders.org.uk \
-d "onlineorders.org.uk,www.onlineorders.org.uk" \
-w /var/www/html/hub.onlineorders.org.uk \
-d "hub.onlineorders.org.uk,www.hub.onlineorders.org.uk" \
-w /var/www/html/admin.onlineorders.org.uk \
-d "admin.onlineorders.org.uk,www.admin.onlineorders.org.uk" \
-i apache \
--keep-until-expiring

--cert-name is far more powerful than --expand

3 Likes

Thank you so much for your help in fixing this problem. Your advice worked perfectly. The only thing that I did differently was to revoke the existing certificate beforehand.

Could I also add that I notice that you replied to me on a Sunday morning and I am extremely grateful for such commitment. I have spent a lot of time on this issue and would not have fixed it without your help.

I did read the certbot documentation looking for a list of command line switches (-d, -i, etc) which might have led me to the solution you provided, but I couldn't find one in the certbot documentation. I am sure that it exists, but I just couldn't find it. I logically looked at the Certbot Commands section here: https://certbot.eff.org/docs/using.html#certbot-commands, a paragraph high up there would be the ideal place perhaps. Just a cheeky suggestion.

Crucially you fixed my problem and I'm very grateful. Many thanks for your help again.

3 Likes

I'm so happy that everything worked out. Your success and praise are the treasures we seek. Your keen observation of my commitment is a rare gift for which I thank you most deeply. :pray: Often a smile is my only reward, which I accept without reservation. If you have any further questions or run into any trouble, you know where to find us. :slightly_smiling_face:

In the future, you don't need to worry about revoking a certificate unless its private key has been compromised. You can just use sudo certbot certificates to find the name of the certificate you want to delete then sudo certbot delete --cert-name *name* to delete it properly. Revocations are like exorcisms: frequently considered, but extremely rarely called for. :wink:

As for documentation of command-line switches, you were almost there... :slightly_smiling_face:

3 Likes

That's the stuff I needed! I have a rare problem of reading things instead of scrolling, but in my defence, there was a lot of reading in between. :slight_smile:

If you put a hyperlink tag somewhere near the top, you'll make a grumpy old programmer very happy.

Thanks again.

2 Likes

To be fair to the certbot devs (since I have been one of the most outspoken critics of their official user guide), you could use the second link from the bottom of the table of contents at the very top of the official user guide.

2 Likes

That is a fair point. I am sure you will know how your brain starts to freeze up when you are dealing with a new problem. Pointed out, that is quite obvious. I work alone and that kind of oversight is the sign of a tired 54 year old mind! Onlineorders has taken me about 5 years to get this far mostly funded by my freelance work. I'm no stranger to long days.

I am Microsoft trained originally but jumped ship to open source about 14 years ago and the only thing I miss is the MS documentation. It is really good. I had to learn php and MYSQL and have written my own js framework for this project, so I am no stranger to support groups either. I have probably spent months of my life going in circles on Stack Overflow. Believe me, I was impressed at the certbot documentation and your help. My suggestion was probably a bit impertinent, but I did mean it in good faith. I should have spotted that link.

I actually found something subsequent to your solution. There was a broken link from the sites-available directory to the 000-default-le-ssl.conf file. I found it using grep. Might be useful knowledge for you. I think that could have been the problem, but I have no idea how the link was broken. I think it might have been when I was building the mail server.

If I find anything else that is relevant I will drop you a line and if I can ever return the favour, please do not hesitate to ask. Take good care and thanks again.

Peter

3 Likes

Very self-trained myself here too. :slightly_smiling_face: You're definitely not alone. :wink: I built my own ACME client from researching all over the place. I'm striving to improve documentation how I can here. Your suggestions (and constructive criticisms) are always welcome and encouraged. We will not take offense in the slightest. It really beats mailing out a questionnaire. I only pointed out the link for your usage.

Thank you for volunteering the information about the broken link. Additional pieces of the puzzle are always welcome in the history.

You are always welcome here Peter. Stop by anytime. :wave:

2 Likes