This can be done from Configurator 2 if and only if you already have the .ipa file.
iTunes used to sync .ipa files back from the phone as part of the backup strategy. This practise stopped some years ago & has since relied entirely on OTA downloads of current versions from the App Store.
It may be possible to use the older & now unsupported last version of iTunes that allowed this, to extract an .ipa from the current phone, which you could then re-sync to a newer phone… assuming it will run on the newer phone.
The link to the last version of this 12.6.x.x is here – Apple KB: Deploy apps in a business environment with iTunes however it appears it will not run in later macOS versions without some effort. See MacRumors: iTunes 18.104.22.168 on MacOS Catalina and also apparently will still work for an iPhone 8, but not an 11 – Reddit: Does iTunes 22.214.171.124 work with iOS 14?
All of this does imply it might be a tougher job than it once was. I cannot test any of this empirically, I’m afraid.
As a long shot, you may have the .ipa buried deep within Time Machine, if it goes back far enough. This was actually how I achieved a similar task a few years ago.