All I did was add a firewall rule to pass ICMP ping request, and firewall rules are reloaded and take effect immediately in pfSense. I didn't reboot or do anything else except go straight to trying the dry run again.
Ahh yes of course. It wasn't a one-time problem though, because the local DNS address was incorrect in the server config. I fixed it and then it was a one-time problem..