Certbot reports "Unable to save files" when files are on NFS

I can't figure out why certbot can't write to the conf file indicated in the log below, except that maybe it's because the file is on an NFS.

The file it says it can't write is /etc/httpd/conf.d/httpd_conf/9999-krobeinteractive-dev_httpd.conf

This is actually a symlink to a file on an NFS mnt:

/etc/httpd/conf.d/httpd_conf/9999-krobeinteractive-dev_httpd.conf -> /mnt/krobeinteractive/config/9999-krobeinteractive-dev_httpd.conf

..where /mnt/krobeinteractive is the mount point of the NFS.

The file can be read/write as expected. For example, I can use vi to edit it and save it:

sudo vi /etc/httpd/conf.d/httpd_conf/9999-krobeinteractive-dev_httpd.conf

I noticed if I change the symlink to point to a conf file on the local disk like below, it works:

/etc/httpd/conf.d/httpd_conf/9999-krobeinteractive-dev_httpd.conf -> /home/krobeinteractive/9999-krobeinteractive-dev_httpd.conf

...where /home/krobeinteractive is just the home dir on the local disk

Or if I just place the actual file right at /etc/httpd/conf.d/httpd_conf/9999-krobeinteractive-dev_httpd.conf, with no symlink, it works.

So I would think that rules out the fact that it's a symlink being the problem, and points to the fact it's on an NFS as the issue.

This dev site is running on a newly installed and yum updated Rocky Linux 9.4 VM, and the NFS is a Windows cifs mount, mounted as follows in /etc/fstab:

//192.168.56.10/dev /mnt/krobeinteractive cifs uid=krobe,gid=apache,file_mode=0774,dir_mode=0775,rw,suid,credentials=/etc/smb_credentials,actimeo=1,mfsymlinks 0 0

I have selinux disabled:

getenforce
Disabled

I've been working around this issue for some time now by moving all my NFS files to the local disk when I need to update my certs, and then copying them back to the NFS when done. But it would be nice if I could figure out how to allow certbot to edit the files as they are.

Any recommendations for figuring out why it can't write?

My domain is: dev.krobeinteractive.com

I ran this command: sudo certbot renew -v

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/dev.krobeinteractive.com.conf


Certificate is due for renewal, auto-renewing...
Plugins selected: Authenticator apache, Installer apache
Renewing an existing certificate for dev.krobeinteractive.com and dev.cms.krobeinteractive.com
Performing the following challenges:
http-01 challenge for dev.cms.krobeinteractive.com
http-01 challenge for dev.krobeinteractive.com
Unable to save files: /etc/httpd/conf.d/httpd_conf/9999-krobeinteractive-dev_httpd.conf.
Cleaning up challenges
Failed to renew certificate dev.krobeinteractive.com with error: Augeas.save() failed: No error


All renewals failed. The following certificates could not be renewed:
/etc/letsencrypt/live/dev.krobeinteractive.com/fullchain.pem (failure)


My web server is (include version): Server version: Apache/2.4.57 (Rocky Linux)

The operating system my web server runs on is (include version):
Rocky Linux 9.4
uname -a
5.14.0-427.24.1.el9_4.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Jul 8 17:47:19 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

My hosting provider, if applicable, is:
none, just a VM on my workstation

I can login to a root shell on my machine (yes or no, or I don't know):
yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 2.11.0

I don't have any insight about the NFS writing part. But, since you have a working system you could switch to using --webroot method. That won't read or write your Apache config like the --apache method does.

Are those two domains in the same VirtualHost?

If so, you could try this:

sudo certbot certonly --dry-run --webroot -w (DocRoot) -d (domain1) -d (domain2)

Where (DocRoot) is the DocumentRoot for the VirtualHost for those domains

And domain1 and 2 are just your two (long) domain names :slight_smile:

The --dry-run is just a test and will not interfere with your existing certs. If it works run the same command without --dry-run

4 Likes

Yes that worked perfectly, just the workaround I needed. Interestingly it works even though both my doc root and my /etc/letsencrypt directory are also on that same NFS. Thank you!

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.