Too many requests of a given type :: Error creating new order ::

Absolutely! Here you go, this is what that command spits out:

/etc/letsencrypt/archive:
total 8
drwxr-xr-x 2 root root 4096 Mar 31 12:02 mail.ryuuzaki.jp
drwxr-xr-x 2 root root 4096 Mar 29 00:05 ryuuzaki.jp

/etc/letsencrypt/archive/mail.ryuuzaki.jp:
total 160
-rw-r--r-- 1 root root 1911 Mar 31 12:02 cert10.pem
-rw-r--r-- 1 root root 1911 Mar 22 01:22 cert1.pem
-rw-r--r-- 1 root root 1915 Mar 22 12:04 cert2.pem
-rw-r--r-- 1 root root 1915 Mar 23 00:01 cert3.pem
-rw-r--r-- 1 root root 1911 Mar 23 12:01 cert4.pem
-rw-r--r-- 1 root root 1911 Mar 24 00:07 cert5.pem
-rw-r--r-- 1 root root 1915 Mar 29 12:02 cert6.pem
-rw-r--r-- 1 root root 1911 Mar 30 00:07 cert7.pem
-rw-r--r-- 1 root root 1915 Mar 30 12:06 cert8.pem
-rw-r--r-- 1 root root 1915 Mar 31 00:04 cert9.pem
-rw-r--r-- 1 root root 1647 Mar 31 12:02 chain10.pem
-rw-r--r-- 1 root root 1647 Mar 22 01:22 chain1.pem
-rw-r--r-- 1 root root 1647 Mar 22 12:04 chain2.pem
-rw-r--r-- 1 root root 1647 Mar 23 00:01 chain3.pem
-rw-r--r-- 1 root root 1647 Mar 23 12:01 chain4.pem
-rw-r--r-- 1 root root 1647 Mar 24 00:07 chain5.pem
-rw-r--r-- 1 root root 1647 Mar 29 12:02 chain6.pem
-rw-r--r-- 1 root root 1647 Mar 30 00:07 chain7.pem
-rw-r--r-- 1 root root 1647 Mar 30 12:06 chain8.pem
-rw-r--r-- 1 root root 1647 Mar 31 00:04 chain9.pem
-rw-r--r-- 1 root root 3558 Mar 31 12:02 fullchain10.pem
-rw-r--r-- 1 root root 3558 Mar 22 01:22 fullchain1.pem
-rw-r--r-- 1 root root 3562 Mar 22 12:04 fullchain2.pem
-rw-r--r-- 1 root root 3562 Mar 23 00:01 fullchain3.pem
-rw-r--r-- 1 root root 3558 Mar 23 12:01 fullchain4.pem
-rw-r--r-- 1 root root 3558 Mar 24 00:07 fullchain5.pem
-rw-r--r-- 1 root root 3562 Mar 29 12:02 fullchain6.pem
-rw-r--r-- 1 root root 3558 Mar 30 00:07 fullchain7.pem
-rw-r--r-- 1 root root 3562 Mar 30 12:06 fullchain8.pem
-rw-r--r-- 1 root root 3562 Mar 31 00:04 fullchain9.pem
-rw------- 1 root root 1704 Mar 31 12:02 privkey10.pem
-rw------- 1 root root 1704 Mar 22 01:22 privkey1.pem
-rw------- 1 root root 1708 Mar 22 12:04 privkey2.pem
-rw------- 1 root root 1704 Mar 23 00:01 privkey3.pem
-rw------- 1 root root 1704 Mar 23 12:01 privkey4.pem
-rw------- 1 root root 1704 Mar 24 00:07 privkey5.pem
-rw------- 1 root root 1704 Mar 29 12:02 privkey6.pem
-rw------- 1 root root 1704 Mar 30 00:07 privkey7.pem
-rw------- 1 root root 1704 Mar 30 12:06 privkey8.pem
-rw------- 1 root root 1708 Mar 31 00:04 privkey9.pem

/etc/letsencrypt/archive/ryuuzaki.jp:
total 160
-rw-r--r-- 1 root root 1899 Mar 29 00:05 cert10.pem
-rw-r--r-- 1 root root 1899 Mar 19 17:28 cert1.pem
-rw-r--r-- 1 root root 1903 Mar 20 00:03 cert2.pem
-rw------- 1 root root 1899 Mar 20 12:04 cert3.pem
-rw-r--r-- 1 root root 1903 Mar 21 00:05 cert4.pem
-rw-r--r-- 1 root root 1903 Mar 21 12:04 cert5.pem
-rw-r--r-- 1 root root 1899 Mar 27 00:01 cert6.pem
-rw-r--r-- 1 root root 1899 Mar 27 12:04 cert7.pem
-rw-r--r-- 1 root root 1903 Mar 28 00:04 cert8.pem
-rw-r--r-- 1 root root 1899 Mar 28 12:08 cert9.pem
-rw-r--r-- 1 root root 1647 Mar 29 00:05 chain10.pem
-rw-r--r-- 1 root root 1647 Mar 19 17:28 chain1.pem
-rw-r--r-- 1 root root 1647 Mar 20 00:03 chain2.pem
-rw-r--r-- 1 root root 1647 Mar 20 12:04 chain3.pem
-rw-r--r-- 1 root root 1647 Mar 21 00:05 chain4.pem
-rw-r--r-- 1 root root 1647 Mar 21 12:04 chain5.pem
-rw-r--r-- 1 root root 1647 Mar 27 00:01 chain6.pem
-rw-r--r-- 1 root root 1647 Mar 27 12:04 chain7.pem
-rw-r--r-- 1 root root 1647 Mar 28 00:04 chain8.pem
-rw-r--r-- 1 root root 1647 Mar 28 12:08 chain9.pem
-rw-r--r-- 1 root root 3546 Mar 29 00:05 fullchain10.pem
-rw-r--r-- 1 root root 3546 Mar 19 17:28 fullchain1.pem
-rw-r--r-- 1 root root 3550 Mar 20 00:03 fullchain2.pem
-rw-r--r-- 1 root root 3546 Mar 20 12:04 fullchain3.pem
-rw-r--r-- 1 root root 3550 Mar 21 00:05 fullchain4.pem
-rw-r--r-- 1 root root 3550 Mar 21 12:04 fullchain5.pem
-rw-r--r-- 1 root root 3546 Mar 27 00:01 fullchain6.pem
-rw-r--r-- 1 root root 3546 Mar 27 12:04 fullchain7.pem
-rw-r--r-- 1 root root 3550 Mar 28 00:04 fullchain8.pem
-rw-r--r-- 1 root root 3546 Mar 28 12:08 fullchain9.pem
-rw-r--r-- 1 root root 1704 Mar 29 00:05 privkey10.pem
-rw-r--r-- 1 root root 1704 Mar 19 17:28 privkey1.pem
-rw-r--r-- 1 root root 1704 Mar 20 00:03 privkey2.pem
-rw-r--r-- 1 root root 1704 Mar 20 12:04 privkey3.pem
-rw-r--r-- 1 root root 1704 Mar 21 00:05 privkey4.pem
-rw-r--r-- 1 root root 1704 Mar 21 12:04 privkey5.pem
-rw-r--r-- 1 root root 1704 Mar 27 00:01 privkey6.pem
-rw-r--r-- 1 root root 1704 Mar 27 12:04 privkey7.pem
-rw-r--r-- 1 root root 1704 Mar 28 00:04 privkey8.pem
-rw-r--r-- 1 root root 1708 Mar 28 12:08 privkey9.pem

/etc/letsencrypt/live:
total 12
drwxr-xr-x 2 root root 4096 Mar 31 12:02 mail.ryuuzaki.jp
-rw-r--r-- 1 root root  740 Mar 22 01:22 README
drwxr-xr-x 2 root root 4096 Mar 29 00:05 ryuuzaki.jp

/etc/letsencrypt/live/mail.ryuuzaki.jp:
total 4
lrwxrwxrwx 1 root root  41 Mar 31 12:02 cert.pem -> ../../archive/mail.ryuuzaki.jp/cert10.pem
lrwxrwxrwx 1 root root  42 Mar 31 12:02 chain.pem -> ../../archive/mail.ryuuzaki.jp/chain10.pem
lrwxrwxrwx 1 root root  46 Mar 31 12:02 fullchain.pem -> ../../archive/mail.ryuuzaki.jp/fullchain10.pem
lrwxrwxrwx 1 root root  44 Mar 31 12:02 privkey.pem -> ../../archive/mail.ryuuzaki.jp/privkey10.pem
-rw-r--r-- 1 root root 692 Mar 22 01:22 README

/etc/letsencrypt/live/ryuuzaki.jp:
total 4
lrwxrwxrwx 1 root root  36 Mar 29 00:05 cert.pem -> ../../archive/ryuuzaki.jp/cert10.pem
lrwxrwxrwx 1 root root  37 Mar 29 00:05 chain.pem -> ../../archive/ryuuzaki.jp/chain10.pem
lrwxrwxrwx 1 root root  41 Mar 29 00:05 fullchain.pem -> ../../archive/ryuuzaki.jp/fullchain10.pem
lrwxrwxrwx 1 root root  39 Mar 29 00:05 privkey.pem -> ../../archive/ryuuzaki.jp/privkey10.pem
-rw-r--r-- 1 root root 682 Mar 19 17:28 README

/etc/letsencrypt/renewal:
total 8
-rw-r--r-- 1 root root 524 Mar 31 12:02 mail.ryuuzaki.jp.conf
-rw-r--r-- 1 root root 499 Mar 29 00:05 ryuuzaki.jp.conf

Hahaha runaway cron job. Yes, that’s what it looks like, is so strange.
I am the only person with access to this server. Perhaps something I did created another user and setup that cron job under that user?
But I always log in with the same user name, or do sudo. I have never logged in or changed my login info.

% sudo -i
# crontab -e

or

% sudo crontab -u root -e

(I see you use zsh, I adapted the prompt)

1 Like

Thank you so much for your help.

Yes I use OhMyZsh with "rkj-repos" theme. Is cute.

So the first command

sudo -I
crontab -e

gives me an empty tmp file.

same with

sudo crontab -u root -e
Just a tmp blank text file:
/tmp/crontab.abEGtz/crontab

where else could guy be hidden?

I used a lowercase i, I don’t know what a uppercase I does.

% sudo ls /var/spool/cron/crontabs
% sudo ls -r /etc/cron*

zprezto with modified steeef here, but not on servers :smiley:

sudo ls /var/spool/cron/crontabs
root ryuuzaki

sudo ls -r /etc/cron*

/etc/crontab

/etc/cron.weekly:
update-notifier-common	man-db

/etc/cron.monthly:

/etc/cron.hourly:

/etc/cron.daily:
webalizer		spamassassin	    ntp      logrotate		apt-compat
update-notifier-common	quota		    mlocate  dpkg		apport
ubuntu-advantage-tools	popularity-contest  mdadm    bsdmainutils	apache2
sysstat			passwd		    man-db   apt-show-versions	00logwatch

/etc/cron.d:
sysstat    popularity-contest  mdadm	    certbot  amavisd-new
postwhite  php		       letsencrypt  awstats

zprezto is nice too, but I don’t like the colored “sales tag” effect. And I slap it everywhere, from my MacBook Pro to all the raspberry pis around the house. Also with Tmux.
Don’t want to sound like I know what I’m doing, cuz I really don’t (nano user here)
Does that output means anything to you? and sorry, yes I did -i but it was autocorrected to capital.

you need to grep those files and see if any have a line calling certbot (grep has a recursive option if you need it grep -r pattern /path/to/directory)

I found omyzsh too heavy (it’s not like zprezto isn’t, but my config at least is) – and my config isn’t that colored :wink:

1 Like

OMG I found it thanks to your comment!
It was here:

/etc/cron.d/letsencrypt

The line reads:

0 */12 * * * root /opt/certbot-auto renew --quiet --no-self-upgrade --force-renewal

But I also have another file:
/etc/cron.d/certbot

with this contents:

# /etc/cron.d/certbot: crontab entries for the certbot package
#
# Upstream recommends attempting renewal twice a day
#
# Eventually, this will be an opportunity to validate certificates
# haven't been revoked, etc.  Renewal will only occur if expiration
# is within 30 days.
#
# Important Note!  This cronjob will NOT be executed if you are
# running systemd as your init system.  If you are running systemd,
# the cronjob.timer function takes precedence over this cronjob.  For
# more details, see the systemd.timer manpage, or use systemctl show
# certbot.timer.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(43200))' && certbot -q renew

What do you think I should remove? Or… how?

keep this, remove the other. (this is installed by the certbot package, I think. Also check systemctl list-timers --all and see if the certbot one is enabled)


wow, it looks so flashy :smiley:

1 Like

but then, this should not work. maybe you have certbot and certbot-auto, but certbot-auto is only in path for root.

1 Like

Right!? But I’m just covering the fact that I lack so much skills by putting so much decorations.

So when you say remove the other. is there a command to take it off? Or just delete the
/etc/cron.d/letsencrypt
file?

systemctl list-timers --all
Doesn’t show the letsencrypt file, but I do still get an email of that command running.

NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
Wed 2020-04-01 18:39:00 JST  14min left    Wed 2020-04-01 18:09:06 JST  15min ago  phpsessionclean.timer        phpsessionclean.service
Wed 2020-04-01 22:24:38 JST  3h 59min left Tue 2020-03-31 22:24:38 JST  20h ago    systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
Wed 2020-04-01 23:30:57 JST  5h 6min left  Wed 2020-04-01 02:10:49 JST  16h ago    motd-news.timer              motd-news.service
Thu 2020-04-02 02:24:35 JST  7h left       Wed 2020-04-01 12:01:58 JST  6h ago     apt-daily.timer              apt-daily.service
Thu 2020-04-02 06:32:28 JST  12h left      Wed 2020-04-01 06:54:11 JST  11h ago    apt-daily-upgrade.timer      apt-daily-upgrade.service
Thu 2020-04-02 08:08:04 JST  13h left      Wed 2020-04-01 17:28:45 JST  56min ago  certbot.timer                certbot.service
Mon 2020-04-06 00:00:00 JST  4 days left   Mon 2020-03-30 00:00:02 JST  2 days ago fstrim.timer                 fstrim.service
n/a                          n/a           n/a                          n/a        snapd.snap-repair.timer     
n/a                          n/a           n/a                          n/a        ureadahead-stop.timer        ureadahead-stop.service

9 timers listed.

there’s a certbot one, it’s running.

should be fine.

you should check if you have something in /opt

2 Likes

inside /opt/
there is

/opt                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
├── certbot-auto                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
└── eff.org                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
    └── certbot                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
        └── venv                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
            ├── bin                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
            │   ├── activate                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
            │   ├── activate.csh                                                                                                                                                                                                                                                                                                                                                                                                                                                              
            │   ├── activate.fish      
            ...

This is usually the moment when I start deleting stuff and then mess up everything even worst.

backup, before.

or move instead of delete.

1 Like

:sweat_smile:
Ubuntu can backup?
So, who should I kill? Or delete?

man tar :wink:

tar -cf ~/filename.tar /directory/to/backup

1 Like

Ok so do that tar backup and then

delete:

/etc/cron.d/letsencrypt

and also delete

/opt/certbot-auto

?

you can delete the entire /opt directory, if the only contents are certbot-auto and eff.org (whose contents are certbot only)

but before you might want to run certbot renew --dry-run to see if your non-auto certbot works.

1 Like

Well that was a lot of red: :dizzy_face:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/mail.ryuuzaki.jp.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Attempting to parse the version 0.35.1 renewal configuration file found at /etc/letsencrypt/renewal/mail.ryuuzaki.jp.conf with version 0.31.0 of Certbot. This might not work.
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for mail.ryuuzaki.jp
Cleaning up challenges
Attempting to renew cert (mail.ryuuzaki.jp) from /etc/letsencrypt/renewal/mail.ryuuzaki.jp.conf produced an unexpected error: Problem binding to port 80: Could not bind to IPv4 or IPv6.. Skipping.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/ryuuzaki.jp.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Attempting to parse the version 0.35.1 renewal configuration file found at /etc/letsencrypt/renewal/ryuuzaki.jp.conf with version 0.31.0 of Certbot. This might not work.
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for ryuuzaki.jp
Cleaning up challenges
Attempting to renew cert (ryuuzaki.jp) from /etc/letsencrypt/renewal/ryuuzaki.jp.conf produced an unexpected error: Problem binding to port 80: Could not bind to IPv4 or IPv6.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/mail.ryuuzaki.jp/fullchain.pem (failure)
  /etc/letsencrypt/live/ryuuzaki.jp/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/mail.ryuuzaki.jp/fullchain.pem (failure)
  /etc/letsencrypt/live/ryuuzaki.jp/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2 renew failure(s), 0 parse failure(s)

this is another issue. you should probably use another authenticator, --webroot or whatever is proper for your http server (--apache or --nginx, but these also install certificates).

don’t forget --dry-run, you are dancing on rate limits.

2 Likes