A random port won’t work. ACME currently offers three possible validation choices
-
Respond to a HTTP (not HTTPS) GET of /.well-known/acme-challenge/stuffChosenByTheACMEprotocol correctly on port 80 of the host to be validated
-
Respond to a fake HTTPS connection that does SNI for an invalid hostname in a particular way on port 443 of the host to be validated.
-
Provision a DNS TXT record in your domain with specified contents
The Standalone mode can do the first two of those options by itself, but as you’ve observed it can’t do this while you have a real web server running. “Webroot” mode lets you achieve the first option, if you can configure a place on your filesystem from which such a response will be served, it will create the file needed, tell the Let’s Encrypt servers it’s in place and wait for them to check it’s OK. You will still need to arrange to any HTTP server reload or whatever.
DNS mode is good if you’ve got programmatic control of DNS for an organisation, but the certbot software doesn’t (last I checked) offer this mode so you need to look for third party scripts that show how it’s done.