Unblock IP address request 145.239.86.144

Hello

We have a mattermost server running as a set of docker containers (https://github.com/mattermost/mattermost-docker) on ubuntu(16) server machine with ip: 145.239.86.144 and with mattermost.iterative.pl domain.

After a year and half of running this setup without any problem today it suddenly failed to renew cert via acme with following msg:

Error initializing issuer: 403 urn:ietf:params:acme:error:rateLimited: Your IP, 145.239.86.144, has been blocked due to ridiculously excessive traffic. Once this is corrected you may request this be reviewed on our forum https://community.letsencrypt.org

After a bit more digging we found out that in directory with certs there are plenty of generated for domains which clearly were not ours, ie:

pkakxmtgtjochtm.vixera.eu+rsa
vixera.eu
vixera.eu+rsa

At the moment we are not 100% sure how that happen, but it seems that version of mm we had been running got some vulnerability, which allowed attacker to manipule mattermost to generate those certs (and as a result flood lets encrypt server with requests). Note that we hadn’t been using reverse proxy, but just mattermost server directly and its internal acme lets encrypt setup along with it.

As a remedy, for now we had:

  • removed all unwanted cert files
  • updated mattermost app to latest version
  • changed config to use nginx as a reverse-proxy along with SSL termination at its stage.
    Due to block we had to manually generate new cert, but obviously ideally we would pref to enable acme on nginx or setup certbot auto-renew.

To our best knowledge remedies we did should stop any excessive traffic from happening further. So could you please unblock our IP address 145.239.86.144?

cheers,
Krzysztof

3 Likes

Ping @cpu : ridiculously excessive traffic

1 Like

Hi @DonCziken,

I’ve opened a ticket with our SRE team to have your IP unblocked. Someone will update this forum thread once the change has been made.

Thanks for addressing the excessive traffic,

3 Likes

Your ip address has been unblocked. You should now be able to issue with Let’s Encrypt!

4 Likes