The camera view randomly changes while I’m in a game

To fix this try changing the “VR Camera Change” setting to Double click.
You will find this setting under the "Control" settings in the Options menu.