Virtual Reality has until now seen limited actual use in society other than in the gaming industry. A reason for this could be its exclusively individual-viewpoint based nature and a lack of possible collaborative experiences together with people with no VR equipment. This study has investigated how joint visual reference points might help a VR and a non-VR user collaborate with each other using a repeated measures design with three conditions. In the experiment, where one user was equipped with a HTC Vive and the other stood in front of a large screen, the pair was presented 0, 4 or 9 joint visual reference points from their own viewpoint. Results of the tasks performed by the participants indicates that 9 joint visual reference points increased a pair’s collaboration efficiency. However, the effect was not present once joint attention had been fully established. Furthermore, non-VR users found it significantly harder giving instructions to the other user when there were no joint visual reference points available while the VR-users did not find it significantly harder to do so. Additionally, differences between VR users’ and non-VR users’ spatial orientation ability were found to predict different patterns over the three conditions. Judging from the results, it seems that for the VR-users, 4 reference points helped more than 0 and 9 helped more than 4. However, an interaction effect was found on the non-VR users between spatial orientation ability and visual reference points condition. 4 reference points had a counter-productive effect on task efficiency for the non-VR users with lower spatial orientation ability while 4 reference points did seem to help the higher spatial ability group. 9 joint visual reference points completely eliminated group differences between high and low spatial orientation ability groups for both VR users and non-VR users.