How does HTC VIVE realize the third perspective in recording virtual games?

1. Switch to the third-person perspective: The ordinary Vive program is to place two virtual cameras corresponding to the right eye at the position of the player's helmet, generate pictures of the left and right eyes respectively and send them to HMD, and then copy one of them to the PC screen. In order to create a game screen with a third-person perspective on the screen, an additional virtual camera needs to be placed at a certain distance from the player, and the lens should be aimed at the player's position. After that, what is displayed on the PC is no longer a copy of the left and right eye pictures, but is drawn separately by this extra camera (which will increase the rendering burden).

2. Record the image of the experiencer: Because we only need the image of the player himself, we don't need to see the background such as tables and walls in the room. So we need to use image cutting technology. When recording, let the experiencer be in a room surrounded by green cloth (or other colors), and then use a camera (network camera or professional camera) to record the video while he is playing the game. Finally, the green color is removed from the captured video, leaving only the human image.

3. Combine the game screen with the camera image: I mentioned a virtual camera and a real camera in 1. Second. They are. In order to make the images they shot together, it is necessary to ensure that their perspectives are the same. That is to say, if the position of the virtual camera in the game is 2m in front of the protagonist's left side, and its orientation makes an angle of 120 degrees with the protagonist's orientation, then it is necessary to ensure that the camera in reality is 2m in front of the experiencer's left side, and its orientation makes an angle of 120 degrees with the experiencer's orientation. This requires positioning and orientation by means of a third handle. Tie an extra handle to the camera, or tie the handle to the same position as the camera. The principle is to ensure that their relative position and direction remain unchanged. In this way, when the camera moves in reality, the game changes the position of the virtual camera by sensing the position change of the handle. Then the objects in the game are divided into two categories according to their positions, one is the foreground, which is located between the camera and the player, and the other is the background, which is located behind the player. Finally, the background, the image of the experiencer and the foreground are combined into a video.

I. Hardware part:

1. Studio: the wall and floor are covered with green cloth, 5m*5m.

2. Camera: Two Solutions

A) Low cost: it is a webcam, such as Logitech webcam C930e.

B) High cost: professional camera+video capture card. For example, Sony A7S II.

3. High-performance host: Because you need to run Vive games and videos at the same time, the configuration requirements are high. The graphics card is recommended to be 980ti, and the CPU is recommended to be above i7-4790.

4. The third handle: At present, Vive can only connect two wireless handles, so it needs a longer USB cable to connect the third handle.

5.4K screen: In order to synthesize 1080p video, the screen needs to be able to accommodate multiple 1080p windows.

Second, the software part:

1.Unity3D+SteamVR: At present, only the programs made by Unity3D support MR mode. As long as the version of SteamVR used in the game is 1.0.8 or above, it can be used for shooting. Otherwise, developers need to upgrade SteamVR before releasing the version.

2.OBS or other video synthesis software.