Tried this with --server, --eab-kid and --eab-hmac-key because I have one cert that uses a different ACME authority than other certs, and therefore has to use different credentials.
When I ran the --force-renew with the last three arguments it did work (didn't without them - I checked), but the credentials still do not appear in /etc/letsencrypt/renewal/www.example.com.conf like I expect them to according to the above message. Is this the right beaviour? Or should I be doing something differently?
If we're talking about the EAB, then yes, you won't find those in the renewal config: those two EAB-settings are only used when your ACME account is created. It's the job of the ACME server to remember the account binding internally on account creation.
That's right: --eab-kid and --eab-hmac-key are only relevant when registering the ACME account for the first time.
For subsequent invocations, all you need to remember is the --server flag, which is included in the renewal .conf file. Certbot has a certbot reconfigure command now which should be used in most cases instead of directly modifying those files or using --force-renewal.
Note that the reconfigure subcommand was added in Certbot 2.3.0. When using snap, I think this is automatically done somehow (personally I would HATE that..), but if you're not using snap, make sure you're upgraded to 2.3.0 or higher.