First, I hope that your credentials are set up so that the user MUST change them during the first login so that those who configure them can not know them anymore.
If a remote office has a trusted administrator, send that administrator an encrypted set of keys / passwords to use once to share with other users. You can simply ask them to use the password # 12 for their initial login.
Depending on your threat model (do you fear state-level actors, ie work with a foreign country office in country 4, which could engage in commercial espionage for the account of your local competitors), is this a classic case? calling someone on a landline is a great solution.
The mere fact of calling someone on a landline and telling them the initial login credentials over the phone works very well.
Beyond that, GPG is still a classic way of doing it, as many people have responded to it. I have some examples of using the public key and using it more securely than the default symmetric use in this answer on Superuser.com.
Depending on your regulatory requirements, the OTR is a method of encrypting communications, especially instant messages (see Pidgin as an example), which also allows authentication by "shared secret" . you can share an easy-to-seize password on the phone when you're on instant messaging to validate that the instant messaging session does not involve a man at the center, or use any aspect of his job that he would be difficult for someone else to master of.
If you already have a way to send an e-mail that you can trust only to your recipient and your own network / e-mail administrator, you can use a "secure messaging" service such as Cisco Registered Envelope Service. or a substitute.
Especially for SSH keys or extremely long and difficult passwords, you can combine these methods. you can encrypt – perhaps using the symmetrical GPG mode (see the link above) – using a secure password, and then pass this password via the phone or another method so that they can decrypt the authentication token / key SSH / certificate / etc.