Pfsense as router -> Netgear Switch -> ...vlan10 (cshost0, cshost1)
vlan10 - 192.168.10.0/24 subnet (aka Homelab subnet)
Pfsense rules
- NAT Port Forward
Interface: Homelab
Protocol TCP/UDP
Source address: *
Source ports: *
Destination address: ! HOMELAB address
Destination ports: 53
NAT IP: 127.0.0.1
NAT ports: 53
Interface: Homelab
Protocol TCP/UDP
Source address: *
Source ports: *
Destination address: ! HOMELAB address
Destination ports: 53
NAT IP: 127.0.0.1
NAT ports: 53
Interface: WAN
Protocol TCP
Source address: *
Source ports: *
Destination address: WAN address
Destination ports: 80 (HTTP)
NAT IP: 192.168.10.40 (traefik external endpoint)
NAT ports: 80 (HTTP)
Interface: WAN
Protocol TCP
Source address: *
Source ports: *
Destination address: WAN address
Destination ports: 443 (HTTPS)
NAT IP: 192.168.10.40 (traefik external endpoint)
NAT ports: 443 (HTTPS)
- Firewall Rules
Interface: Homelab
Action: Reject
Protocol: IPv4 TCP/UDP
Source: *
Port: *
Destination: *
Port 853 (DNS over TLS)
Gateway: *
Interface: Homelab
Action: Pass
Protocol: IPv4 TCP/UDP
Source: *
Port: *
Destination: 127.0.0.1
Port: 53 (DNS)
Gateway: *
This is associated with the NAT rule above.
- DNS Resolver
Enable: True
Listen Port: 53
Network Interfaces: All
Outgoing Network Interfaces: All
- pfBlockerNG
Enable: True
DNSBL: Enabled
Kubernetes cluster
cshost0 - master node (also a worker) - 192.168.10.2
cohost1 - worker node - 192.168.10.3
DNS server used by both - 192.168.10.1
MetalLB - bare metal load-balancer
csweb namespace:
traefik deployment:
images: traefik:latest, thomseddon/traefik-forward-auth:2
traefik service:
external endpoints - 192.168.10.40:80, 192.168.10.40:443
cert-manager namespace:
cert-manager deployed from jetstack/cert-manager helm chart
# issuer.yaml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-production
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: myemail
privateKeySecretRef:
name: letsencrypt-production
solvers:
- dns01:
cloudflare:
email: myemail
apiTokenSecretRef:
name: cloudflare-api-token-secret
key: api-token
# mycertificate.yaml
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: example-certificate
namespace: namespacethatneedsit
spec:
commonName: '*.example.com'
secretName: example-certificate
dnsNames:
- example
- '*.example.com'
issuerRef:
name: letsencrypt-production
kind: ClusterIssuer
When applying mycertificate.yaml, this is what shows when describing the certificate in the Kubernetes cluster.
status:
conditions:
- lastTransitionTime: '2020-12-05T13:45:10Z'
message: Issuing certificate as Secret does not exist
reason: DoesNotExist
status: 'False'
type: Ready
- lastTransitionTime: '2020-12-05T13:45:10Z'
message: Issuing certificate as Secret does not exist
reason: DoesNotExist
status: 'True'
type: Issuing
At the same time, I checked CloudFlare, and the TXT record _acme-challenge was successfully created with a TTL of 2 minutes.
Running this command "host -t txt _acme-challenge.example.com" outputs:
Host _acme-challenge.example.com not found: 3(NXDOMAIN)
But if I create another TXT record just like it, I can immediately resolve it on any system regardless of DNS server.
I'm rediagnosing this as I write this up, but what's interesting to me is if I make another TXT record in the CloudFlare portal, it does show up instantly using the above command. So cert-manager is able to create the TXT record, but neither cert-manager nor I can resolve the record? But by turning off my firewall rules above and setting my DNS servers to 1.1.1.1, this TXT record can be validated by cert-manager and certs are issued.
This just gets weirder and weirder. Has all this information helped?
