Why can Pixel 4 capture the Milky Way with night shots? Google's long exposure, photo synthesis and scene recognition

The new mobile phone Pixel 4 released by Google has a lot of upgrades, including not only radar-induced gesture operation, but also the camera function of Pixel 3, which can even shoot the whole starry sky outside the night scene. Google Pixel 4 camera is equipped with a dual-lens module, which strives to achieve the effect of "what you see is what you get" and minimize the color difference between photos and objects. Why on earth can Google Pixel 4 shoot the Milky Way? Google recently released a technical article about the functions of the Pixel 4 camera. As for the powerful Pixel 3 for night shooting, please refer to: Analysis of the principle of Google Pixel camera. Why are Pixel 3 and Pixel 3a single-lens cameras so strong?

Extend the exposure time 15 photos. Usually, when a mobile phone shoots the night sky, because the starlight is too weak, it is difficult for the camera of the mobile phone to capture the light, resulting in only darkness in the photo. Therefore, in addition to increasing the aperture, Pixel 4 also lengthens the upper limit of exposure time (that is, extending the shutter time), so that Pixel 4 can have a long exposure of up to 4 minutes, and the lens has enough time to capture weak light spots. Through software update, pixel 3 and pixel 3a can have an exposure time of 1 minute.

▲ The larger the aperture, the more light enters the lens. Conversely, the smaller the aperture, the less light can enter the lens. The numerical value of the aperture size is expressed by the value of f, because the aperture value is in the denominator, the larger the aperture, the smaller the numerical value. As shown in the figure, the aperture of f 1.8 is larger than f 1 1. (Source: Wiki)

Although long-term exposure can capture starlight by the lens, long-term exposure will also encounter many problems, such as overheating of the photosensitive element, unstable hand-holding, or the starry sky moving with time. , each of which will affect the final rendering effect of the photo. Among them, when CMOS (photosensitive element) is overheated, there will be "hot pixels" in the photo, and red dot noise will be generated where there is no light spot. The high sensitivity of the lens will also make the photo prone to noise.

Explanation of terms: sensitivity ISO sensitivity, also known as ISO, is the sensitivity to light, and high ISO is helpful for shooting under the condition of insufficient light and no flash.

▲ Long exposure or high-sensitivity shooting is prone to noise. Just like the obvious fuzzy particles on the left in the above picture, the photo is not clear. (Source: Cat at Home)

In order to solve the noise problem, Pixel 4 does not only take one photo, but divides the photo into multiple frames within four minutes of exposure, that is, takes many photos, so that the exposure time of each photo is not so long. These frames have short exposure time and few blurred parts. Pixel 4 identifies each other by comparing photos, and uses an algorithm to cover the noise in order to get a clear final result.

Keep the star still. Maximum exposure per frame 16 seconds? Because of the long exposure and shooting time, the shooting and post-synthesis of 1 clear Galaxy photos Pixel 4 takes about 4 minutes. However, both the moon and the stars will move. If the shutter time of a single photo, that is, the exposure time is too long, it may not be the stars, but the trajectory of the stars. In order to avoid shooting the orbit of the star, the exposure time of 16 seconds per frame is enough for Pixel 4 to capture enough light and make the star look like a light spot.

▲ Because the stars are constantly moving, if the exposure time is too long, you can't shoot a spot, only light. The picture shows the blurred starry sky in a single frame exposed for 2 minutes. (Source: Google)

Of course, if you want more detailed photos, Pixel 4 can choose to take a few more frames, and there are more materials to make the photos clearer. However, shooting is related to the photographer's patience. Google said that few people are willing to wait for an image for more than 4 minutes, so Pixel 4 limits a single photo to at most 15 frames, and each frame is at most 16 seconds.

Machine learning and image training automatically distinguish between scenery and starry sky. In order to avoid overexposure of scenery other than starlight, Pixel 4 will automatically distinguish between starry sky and scenery through machine learning and training of more than 6.5438+million images, and automatically overlap in different degrees. Pixel 4 can identify sky and non-sky blocks, and selectively increase the contrast and brightness of the blocks, so that every part of the photo has the correct color. So pixel 4 is superimposed on the starry sky, which will not affect the brightness of other scenes and portraits at the same time.

▲ If the starry sky and the scenery are in the same photo, Pixel 4 will automatically distinguish the sky from the scenery and make different treatments to avoid the scene being too bright or the starry sky not being clear enough. (Source: Google Photos, Nicholas Ray Wilson)

The milestone of mobile phone night shooting still has shortcomings to be solved. Although not as good as professional cameras, mobile phones can clearly shoot night scenes, starry sky and galaxies, which is a great technological innovation. However, although Pixel 4 is very powerful in starry sky shooting, Google also said that there is still room for improvement. It can be improved in noise judgment, high brightness contrast shooting, and strict environmental standards (extremely dark, fixed tripod, long countdown) to shoot ideal effects. But even so, the Galaxy shooting function of Pixel 4 mobile phone camera is amazing, and I look forward to more breakthroughs in Google mobile phone camera in the future.

?