Did artificial intelligence blow up the camera of the mobile phone?

Artificial intelligence is exploding the camera of mobile phone.

Since the birth of mobile phone AI chip, this series has tracked, analyzed and reported many new changes in the field of mobile phones. But if I sum up where the biggest change happened after the collision between AI and mobile phone, there may be only one answer: the camera.

From the technical logic point of view, the mobile phone equipped with AI arithmetic processing unit, combined with the image processing ability of its own GPU, has the most significant improvement in image content generation, recognition and processing. Whether it is beauty, beauty or beauty video, the algorithm can give you beauty.

But in fact, high-end machines must embrace AI in unison, and there may be deeper industrial contradictions and practical problems. Mobile phone AI brings more than just a simple improvement of "taking pictures looks better".

We will quote Huawei Mate 10, iPhone X and Google Pixel 2, three smart phones with AI capabilities and computing units, to discuss the changes that AI has brought to the camera and shooting functions of mobile phones.

At one time, we thought that mobile phones could only make phone calls and send text messages, and then with a boom, mobile phones were blown into today's "universal" smart phones. With the arrival of AI, the camera may be experiencing a similar explosion.

Mobile photography has entered the deep water area, and only AI can create differences.

Why did flagship machine, the mainstream of Apple, Google and Huawei, announce to enter the AI era in just a few months?

The most direct reason is obviously that it is inevitable to have a big battle between high-end cameras every year, and now it has entered an extremely tragic stage of bayonet seeing red.

Some people think that Moore's law is close to failure in the optical hardware and camera unit of mobile phones. With the camera with double lens, large aperture and10 million pixels or more becoming standard, the core unit community of the camera seems to have solidified. However, mobile phones should be produced every year and improved every year. What is even more frightening is that for high-end machines, the camera part is absolutely bloody and must not be left behind-it is also a headache for developers and product managers behind mobile phones.

Even Apple, the leader, has been struggling with cameras in recent years. Even if you open some photos of professional evaluation websites, ordinary users may not be able to tell the difference between two generations of models and competing models.

After all, not everyone is a photographer and an artist with such fine resolution. Therefore, in order for consumers to continue to perceive that camera capabilities are changing, they must find a way outside the camera unit-so AI seems to be the best choice.

The introduction of artificial intelligence technology may mean that mobile phones can rely on changes in basic hardware and a large number of algorithms to help improve the shooting and recording experience. Create more gameplay and simpler experience, and get rid of the endless entanglement in a few fields such as aperture and pixel. For mobile phone manufacturers eager for new consumption growth points, it is definitely a surprise.

More importantly, there are many modes and methods for the application of AI computing power and algorithms in mobile phone shooting. It is hard to imagine that the flagship opportunity of secret research and development hits the details of AI applications. This can already be seen in the differentiation between Huawei Mate 10 and iPhone 8/X, and this is just the beginning. After more technologies and development ecology mature, the gap between the two flagship machine models will only widen. This is what mobile phone manufacturers want.

From the bitter sea of homogenization competition, I am afraid I can only rely on AI. So, AI came. ......

Decisive Battle: Algorithms and Hardware Capabilities for Filling Special Scenes

Of course, AI may change the shooting war of mobile phones. This by no means means that this battle will not continue.

In fact, before starting a new gameplay, the last battle that AI helped mobile phones shoot will still focus on the battlefield that mainstream mobile phones have been attacking for many years: special scene shooting.

The so-called special scenes here include night scenes, strong light, dark light and dynamic snapshot, and the specific application examples are varied. Because the camera of the mobile phone is destined to have no depth, it can't handle the imaging effect in extreme light environment like a SLR camera. Therefore, shooting in dark and bright light environment has gradually become the basic element to evaluate the camera ability of a mobile phone.

Look at DxOMARK's ranking of mobile phone shooting results, and you will find that the top five have been basically occupied by AI mobile phones. The important bonus of these mobile phones' photo-taking ability comes from the capture and balance deduction of the light environment by the algorithm. For example, Google Google Pixel 2 will use complex algorithms, such as splitting the information collected by the camera and module deep learning, to achieve spiritual effects and redraw the light source. So we got high marks in white light processing.

In this evaluation chart of DxOMARK, we can see that with the help of AI algorithm, the relationship between light and shadow in flagship machine has been obviously improved compared with the previous pure optical mode, especially in the matching of light and shadow structure.

The algorithm capture technology is not only used for still photography, but also reflected in the realization of ultra-high-speed focusing in high-speed moving environment and recognition+shooting in dark environment.

AI+ mobile phone shooting can also achieve dynamic beauty and video beautification under complex movements and special light. Using the underlying hardware+algorithm, the smart phone can achieve many functions that cannot be achieved by the camera alone.

But this is just a promotion. The real significance of the arrival of AI is to change a lot of so-called "common sense" … ..

The savior of straight men: liberating mankind with the recognition ability of AI.

Please assume that you are a straight man who doesn't like taking pictures, but loves his girlfriend ... what's your first feeling when you take pictures of your girlfriend with an expensive mobile phone?

Are you happy? Awkward? No, your first feeling is that it's too complicated and you're tired. You will find that there are many functions on the mobile phone, such as HDR for a while, beauty and skin care for a while, large aperture to blur the background for a while, and depth of field mode for a while ... If you catch someone with a bad temper, you will either throw your mobile phone or throw your girlfriend.

In fact, such complicated operations and proper nouns will be produced. Because in order to deal with different shooting objects and scenes, mobile phones must imitate the shooting mode and give different exposure, white balance and focusing strategies. But more functions mean more complicated operations, and there are still more disabled people in the world. Can this step be omitted?

Perhaps one of the most exciting things about AI joining mobile phones is that one of AI's main abilities is to recognize and understand images. In other words, automatically triggering the corresponding function through the recognized image information may mean that the straight men who are incompetent in taking pictures can be liberated.

For example, Google's Pixel 2 makes good use of AI's recognition ability to take pictures. In its camera unit, a neural network algorithm is hidden, which is specially used to distinguish the information of people in shooting. Then it is transmitted to the back end for corresponding phased image processing.

Of course, the main purpose of Google's technology is to make the camera get the ability to blur the background of dual-camera beauty photos. And Huawei Mate 10' s recognition+shooting, that is, its propaganda "smart photography" mode seems to be a more radical solution. The processing method of Mate 10 is to learn the information characteristics of pictures in advance through NPU, so as to obtain the ability to classify the photographed objects. Then organize it into 13 photo mode.

When the camera triggers this mode when shooting, the phone will automatically adjust to the corresponding mode.

In fact, this mode of recognizing before shooting is likely to become a universal standard in a short time, and even replace some commonly used functions. After all, in our daily life and social life, the purpose and expected effect of taking pictures are highly consistent. Summarizing these characteristics through the deep learning ability brought by AI and intelligently fulfilling users' expectations may be the normal state of future photography.

Maybe in the near future, when you go to the seaside, it's raining, and there are no big stars in Hollywood, you can use AI to create blue sky, white clouds, green mountains and green waters, and even occasionally get profiles of several stars. .....

Of course, the recognition and shooting ability of AI should really be liberated by the elderly, children and most ordinary people who only occasionally take pictures with their mobile phones. After all, most of us don't have time to study complicated courses and hone our superb aesthetics, but we still hope that the carefully prepared moments can leave more beautiful memories.

AI makes the world fairer, maybe.

Big bang that has happened: cameras are not just used to take pictures.

Two visible AI+ camera modes are introduced above: AI lifting shooting and AI changing shooting. AI also gives the camera another kind of empowerment: not just for shooting.

At present, there are two ideas about this function: the first one, represented by iPhone X, makes a fuss about the perception ability, and combines the camera with the sensing component to produce new experiences and functions.

In addition to the new photosensitive elements, the iPhone X camera also adds a rather complicated sensor system. Especially on the front camera, the original depth camera in Liu Hai is used to realize biometric identification and dynamic capture, and special functions such as Animoji and portrait light effect are realized.

In addition, Face ID is also recognized by similar functional structure light. The floodlight sensor can also help users unlock their phones in dark light. This idea may mean that in the future, cameras will not only "see" but also "perceive" the outside world. At present, the functions are only face recognition and some special functions, but it should not be far to develop more functions by using hardware and algorithms.

The other mode is to obtain data learning and agent growth ability through the machine vision ability of the camera itself. AI camera with memory and learning ability not only records image information, but also completes the learning and understanding of this information locally. So as to adapt to the habits of users and achieve a better shooting experience.

Both of these methods mean that the mobile phone AI brings not only the improvement of the camera ability itself. On the contrary, the camera is one of the information collection methods and is incorporated into a larger capacity system. It is very important for the future to cultivate the ability of accurate identification, multi-data processing and content output of mobile phones.

Perhaps when developers start to think collectively about what these functions mean, the real value that mobile artificial intelligence brings to the image world will be revealed.

AI can change many aspects of mobile phones, but fundamentally, what it wants to change is the way of human-computer interaction that is regarded as common sense.