This Mastodon Docker guide that I been doing with the postgresql method was fine until this problem at hand:
Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
Domain: 1renner1.uk
Type: unauthorized
Detail: 2606:4700:3036::ac43:818b: Invalid response from http://1renner1.uk/.well-known/acme-challenge/vZzb6YxU80dNknXLhM27YPFIoJQrXdlYTIiraWdbHXs: 522
My ports 80 & 443 are opened, while 3000 & 4000 are filtered. My containers are running with no errors.
I did read a bit about the reverse proxy problems, but unless the guide was wrong, I'm not sure what I had misunderstood.
(Most common cause) Cloudflare IP addresses are rate limited or blocked in .htaccess, iptables, or firewalls. Confirm your hosting provider allows Cloudflare IP addresses.
My router's firewalls did not block them, nor does the Mastodon server since I haven't done the firewall section near the end of the guide.
An overloaded or offline origin web server drops incoming requests.
According to the Browser's Dev Tools, connection: keep-alive is correct.
The origin IP address in your Cloudflare DNS app does not match the IP address currently provisioned to your origin web server by your hosting provider.
I think I done my Cloudflare DNS correctly. The private IP address of my Mastodon server is the same as my Ubuntu Virtual Machine.
Type A, 119..., Proxied, TTL Auto
Type A, 192..., DNS only, TTL Auto
Also, I did not used any Cloudflare Tunnel or Origin Certificates into my server.
Cloudflare's SSL/TLS was set to Full Strict mode.
I'd turned on & off 'Always use HTTPS' to see if it will make a different. And it didn't.
If you are using Cloudflare Pages, verify that you have a custom domain set up and that your CNAME record is pointed to your custom Pages domain.
I am not using any CF Page.
I read some threads in Cloudflare Community regarding Let's Encrypt, but honestly, I don't know what I readed. Something about adding rules to .well-known/acme-challenge/*? Pause, then after do X, un-pause Proxy and go Full Strict?
The IP should be the PUBLIC IP for your server. Not a Private one.
Can you reach your public IP from the public internet? Like from a different device, perhaps a mobile phone with wifi disabled, try reaching
http://public-ip/
or
https://public-ip/
Because the 522 means a connection failure from the Cloudflare Edge to the public IP you setup as proxied.
Getting this working is probably better handled at the Cloudflare community.
No requests work to your origin server. You need to have a working comms setup before being able to use a Let's Encrypt HTTP Challenge to get a cert.
# This works as Cloudflare handles HTTP request and redirects it
curl -i http://1renner1.uk
HTTP/1.1 301 Moved Permanently
Location: https://1renner1.uk/
Server: cloudflare
# Following that redirect for your "home" page fails
curl -i https://1renner1.uk
HTTP/2 522
server: cloudflare
# HTTP requests like the ACME Challenge fail in same way
# Looks like you setup a page rule to exempt redirecting these to HTTPS
curl -i http://1renner1.uk/.well-known/acme-challenge/Test404
HTTP/1.1 522
Server: cloudflare
For page / cache rules and the ACME HTTP Challenge below is a good reference. But, that isn't your immediate problem. The 522 for HTTPS "home" page indicates the basic comms problem. Note linkp is a MVP at the Cloudflare community:
@MikeMcQ
The IP should be the PUBLIC IP for your server. Not a Private one.
I'm not sure what I have misunderstood. In this context, Cloudflare "reverse proxy" my IP Address (public & private), so when I nslookup 1renner1.uk, it will only show ipv4 & 6 from Cloudflare.
The Private IP was assigned to my Ubuntu VM, so it is the DNS.
Can you reach your public IP from the public internet? Like from a different device, perhaps a mobile phone with wifi disabled, try reaching
No, because it just my Public IP which can be found with 'whatismyip'.
Note linkp is a MVP at the Cloudflare community:
How to fix this error? action parameters are required for the set_config action
Everything else in the options were at default (aka disabled).
With Cloudflare proxy "regular" people first connect to Cloudflare Edge with one of the IP in the DNS (usually two IPv4 and two IPv6).
But, the Cloudflare Edge makes another connection to you across the PUBLIC internet and so must use a public IP. Private IP are only useful solely within a private network.
You could do http://(public-ip) to reach your server directly though. It is not the same as when Cloudflare reaches you but it might show whether a connection is possible.
Or, you could make a DNS-Only entry in Cloudflare like direct.1renner1.uk and set an A record for your public IP. Then try http://direct.1renner1.uk
As for the Cloudflare panel error, you'd have to ask on the Cloudflare forum or perhaps @linkp will respond here
The public IP shown with curl https://ifconfig.io was indeed the same as the previous IP that was proxied.
So, by NOT proxy my public IP; and wait about an hour; the certbot error was changed to this:
Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
Domain: 1renner1.uk
Type: dns
Detail: DNS problem: looking up A for 1renner1.uk: DNSSEC: Bogus: validation failure <1renner1.uk. A IN>: signature crypto failed from 108.162.194.175; no valid AAAA records found for 1renner1.uk
...I don't get it. I thought as long as I have at least one A or AAAA, then its fine, isn't it?
Type A, 1renner1.uk, 192..., DNS only -reserved IP
Type A, 1renner1.uk, 119..., DNS only
CNAME, www, 1renner1.uk, Proxied
Interesting. I do not know why it is not necessary to tell Cloudflare my VM or server private IP, but something did changed again.
Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
Domain: 1renner1.uk
Type: connection
Detail: 119.160.163.29: Fetching http://1renner1.uk/.well-known/acme-challenge/AF5vJuNsCsP8cxdMhPZU4azlSD6HjlFviCAD4DjsaTI: Timeout during connect (likely firewall problem)
I also tried disabling "always https"; and I figured out the Cloudflare Panel error with "ACME Challenge", I just need to "add" the options, but leave the default 'off'.
Let me see if I can figured this one out with this clue.
nmap -p80,443 -Pn 1renner1.uk
PORT STATE SERVICE
80/tcp filtered http
443/tcp filtered https
They were opened at my initial post, but something changed I guess?
There are no other server or app listening to port 80 using fuser 80/tcp, only nginx as planned. My router's firewall doesn't blocked port 80.
Could it be that the Cloudflare's Full Strict mode caused this?
No, the ports on your Origin server were not working on your initial post. At that time you had Cloudflare proxy active so were reaching the Cloudflare Edge. The 522 indicated a problem reaching your Origin server.
We focus on helping people get and configure Let's Encrypt certs. While we often help with immediately related problems yours is a general network setup problem. I am not sure which forum is best for that. But, at least for understanding how a Cloudflare Proxy works review this: Proxy status · Cloudflare DNS docs
Perhaps some other volunteer here will help but I've explained this several times so it is not worth me trying to explain again. Maybe another volunteer's explanation will work better
That said, here are a couple tidbits
When proxied if you set Always SSL your Origin server will only see HTTPS requests on port 443 from the Cloudflare Edge. You are not proxied now so that doesn't matter. But, you wondered if this would break things. And, possibly yes if your Origin is not ready for HTTPS(443). That is what the linkp article helped. And, no, it would not permanently damage your Origin comms setup. It only applies when proxied.
Make sure your ISP allows inbound requests on port 80 and port 443. Some residential ISP do not. I have never seen a valid request get to your server. Not on port 80 and not 443. You may be able to reach things from your own local network. But, requests from the public internet have not worked.
Also make sure your Docker port mappings are correct.
Once you have HTTP and/or HTTPS working for your domain you could try getting a Let's Encrypt cert. Or, even use a Cloudflare Origin CA cert (might be easier for you). Origin CA certificates · Cloudflare SSL/TLS docs
My ports 80 & 443 were indeed, not opened. However, I do not want to expose my machine & server.
So, I correctly configured the Cloudflare Tunnel (the public host's service must be https:192...); which get me a CNAME, so I created A for CNAME Flattening; then I issued a certification with let's encrypt; and bang, it works! (That took more work than I thought.)
The Docker mappings were unchanged; and I didn't download Cloudflare Origin cert into my server (at least, I didn't do it manually).
Although these are more of a DNS & networking problems, I still would like to thank @Nekit and especially @MikeMcQ for the helps.
What do you need the Let's Encrypt cert for when using the Tunnel? I thought the Tunnel was already secure comms between the Cloudflare Edge network and your location (the one running cloudflared).
From the cloudflared device on your location to other servers at your location isn't HTTP satisfactory?
You are right this has mostly been network and DNS. But, I am curious why you'd need an LE cert with a Cloudflare Tunnel. I don't use one myself and only know it from its docs and my general knowledge of similar kinds of connections.
No other reason than once the tunnel was established, and rather than trying to include a CF Origin Certificates into my server (& other stuffs), it was just 1 command away to get a certification (it's where I last left off following the guide).
I did read about the difference between CF SSL & LE, but I don't have a full understanding if LE was necessary or not in my case.
The Tunnel was established in my Proxmox VE's Cloudflare LXC, rather than the VM which hosted the Mastodon server (if that's an option). And I figured if in the future I make more VM + servers, then this current tunnel should take care future port 80 & 443 issues.
