I think @MikeMcQ is on to something.
Please check the your system doesn't have multiple versions of certbot(-auto).
which certbot
which certbot-auto
2 Likes
Thank you this was the clue I needed. I found the problem. The author of the script is doing this for some unknown reason which I'm about to ask.
root@mgdwp_denver-carpet-32:~# grep -rnw '/var/log' -e 'certbot-auto'
/var/log/auth.log.1:1726:Oct 19 04:26:30 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
/var/log/auth.log.1:3567:Oct 20 04:26:27 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
/var/log/auth.log.1:5275:Oct 21 04:26:47 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
/var/log/auth.log.1:6868:Oct 22 04:26:25 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
/var/log/auth.log.1:8472:Oct 23 04:26:25 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
/var/log/auth.log.1:10389:Oct 24 04:26:32 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
/var/log/auth.log:1879:Oct 25 04:26:30 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
/var/log/auth.log:3587:Oct 26 04:27:20 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
/var/log/auth.log:5317:Oct 27 04:26:26 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
/var/log/auth.log:7184:Oct 28 04:26:22 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
/var/log/auth.log:9039:Oct 29 04:26:58 localhost sudo: root : TTY=unknown ; PWD=/root ; USER=root ; COMMAND=/usr/bin/snap alias certbot certbot-auto
1 Like
Looks like a retrofit "catch-all" attempt.
[which actually missed two: "letsencrypt" & "letsencrypt-auto"]
Still. Are there any of those other programs laying around?
2 Likes
yes, it's living in /snap/bin/certbot-auto
wait, it is, but its a symlink to /snap/bin/certbot which is a symlink to usr/bin/snap
Ok. What about?:
which certbot
which certbot-auto
2 Likes
root@wpms-shost1:~# which certbot
/snap/bin/certbot
root@wpms-shost1:~# which certbot-auto
/snap/bin/certbot-auto
root@wpms-shost1:~#
Is it strange for me to wonder... why not find replace certbot-auto -> certbot. Is there some purpose for backwards compatibility w/ 16.04... 18.04 that I'm not clued into or does this just really look like a short shortcut?
I'm not familiar with /snap/bin/certbot-auto
That must have been manually added by a previous admin of that system.
And, yes, it seems like a "short shortcut".
Anywho, that doesn't seem to be part of the problem...
2 Likes
So this leads me to my next theory... and kind of wanting to post the whole story.... Which I guess I could edit out the uninteresting parts for sanity.
The script seems to use the same block for initializing new domains as it would a renewal, so it's hitting the API over again, in my opinion somewhat unnecessarily. I'm going to search my 18.04 boxes for similar errors to see if it's really a 20.04 issue. Right now my working theory is the guys who wrote this are chalking it up to 20.04 is new yadadada we're not supporting it. Which is fine, but still why would the endpoint kill the session. Unless, 20.04 has the new kernal with a really efficient network stack. The majority of users are in commercial LAAS providers with lots of limits on IO, lots of CPU steal so these scripts are executing at a limited clip.
Now I have zero restraints, no IO limits, no CPU steal, its the next best thing to bare metal. My theory here is that it's running quick enough that these unnecessary loops are hitting the endpoint too fast and it's closing the session out, either because it isn't fast enough to respond or it's rate limiting my IP.
I'd have to prove this theory somehow...
1 Like
If you can alter the coding...
Throw in a small delay (just a few seconds) here and there.
See if that normalizes things.
2 Likes
Yah, that is something I can do. I'll try that and also spin up an unedited version and run the processes together and see what happens.
I'll report back when I have something new. Thanks' for the time and help. Much appreciated!
1 Like
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.