camera – Why are portrait photos sometimes sideways?

When I take a portrait picture, it appears correctly on the phone. But when I transfer it to some software, it appears inverted. Why is it? Is it the fault of the phone or software that reads the image? Is there a way to fix a batch of these images so that all software displays them correctly?

The phone is an LG G3, but it has happened on several phones and many people have reported this problem, for example:
https://androidforums.com/threads/vertical-photos-rotating-to-horizontal-orientation.588960/