Can AR HUD replace the central control screen?

Ten years ago, when Elon Musk first introduced the Tesla Model S central control screen, everyone present knew that a new era was coming, because the last iPhone used a screen to subvert the physical keyboard. The power of "intelligence" has been perfectly proven with practical actions. Moreover, Tesla's central control screen not only replaces the physical buttons in the car in form, but also subverts the interactive experience in the car cockpit. Since then, "big screens" have become a standard feature of smart cars.

Even today, 10 years later, car manufacturers are still persistent in their pursuit of “screens”. Huawei's latest AITO Wenjie M5 is equipped with a 10.4-inch curved full LCD instrument panel and a 15.6-inch semi-suspended central control screen. In addition to the 14.6-inch instrument screen and 16.9-inch central control screen, Gaohe HiPhi X is also equipped with a 19.9-inch large passenger screen. The latest Ideal ONE even achieves "four-screen linkage". In addition to new players in car manufacturing, traditional car companies are also working hard on large screens. BMW's latest iDrive 8 uses the largest and clearest curved floating screen in BMW's history. The screen equipped by Mercedes-Benz in its electric flagship model EQS reaches 1.4 meters! You must know that the width of this car is only 1.9 meters.

However, Tesla, the pioneer of the "big screen" era, has become conservative in its attitude towards in-car screens. In its latest Model S Plaid version, it only installs a screen that can "left and right" Mobile” 17-inch screen. This makes people wonder, is Tesla "unable to roll up"? Or does Elon Musk think technology can make another breakthrough?

During the 2021 Shanghai Auto Show, Huawei not only demonstrated its stunning four-seater autonomous driving technology, but also released its latest HUD technology that can project a 70-inch high-definition image at a distance of 7.5 meters. Car companies struggling with in-vehicle "big screens" see new hope.

HUD (Head Up Display), also known as the head-up display system, is evolved from the optical sight technology of fighter jets. It can project vehicle information on the front windshield of the vehicle so that the driver does not have to lower his head to check. Instrument information can keep your eyes on the road ahead to ensure driving safety. With the development of technology, vehicle HUDs have gradually evolved from being able to only display basic information such as speed and reminder icons to a stage where the display range is larger, the picture is clearer, and it can completely replace the instrument panel.

The advancement of autonomous driving technology can free the driver or car occupants' attention from the road, which has put forward higher requirements for the entertainment function of HUD, and also prompted HUD to have a larger screen. Displays are developing in a clearer direction, and the Huawei HUD mentioned above is this route.

Another requirement that the development of autonomous driving technology places on HUD is integration. The autonomous driving system must sense the surrounding environment and plan a driving route in the established virtual space. Andrej Karpathy, director of Tesla's artificial intelligence department, proposed at the Tesla AI Day in 2021 that the virtual space established by the autonomous driving system needs to achieve "spatial fusion", that is, the positions of all objects on the road recognized must be correct. And "time fusion", which means there can be no delay.

The virtual space created by the autonomous driving system is usually displayed on a large screen in the car for the driver to supervise. However, when the driver's sight is off the road, the meaning of supervision is lost. Therefore, the integration of virtual information and the display world on HUD becomes the best solution, which gave birth to AR-HUD.

AR-HUD began to receive widespread attention when the new Mercedes-Benz S-Class was launched in early 2021, and the arrows floating around became a must-have demonstration function for major car companies. At the just-concluded 2022 CES exhibition, AR-HUD has also become a new track for major manufacturers to compete.

Panasonic Motors has released AR-HUD 2.0 with eye-tracking technology, which can optimize the eye distance according to changes in the driver's gaze point, ensuring that no matter how the driver moves his head and line of sight, he can enjoy clear, Accurate AR navigation effect. Qualcomm Snapdragon Automotive Cockpit Platform will introduce Phiar's computer vision and spatial AI technology to implement an in-vehicle AR HUD system that can perceive the environment and navigate. CY Vision released the next generation AR-HUD with 3D display, dynamic focus display, eye tracking, AR image simulation of human eye visual effects, large field of view, high brightness, suitable for various distances and weather conditions, and announced a partnership with BMW Collaborate to develop the latest AR technology. EyeLights and AGC have joined forces to introduce industrialized AR functionality to a series of vehicles, claiming to be able to project up to a 550-inch virtual screen at a projection distance of 50m.

Although AR-HUD has started the "roll" trend again at CES 2022, AR-HUD is not easy to implement technically. You may often see this kind of magic video that uses "borrowed position", but if you are on the scene, this "magic" will be easily exposed, because compared to the "one-eyed" camera, two human eyes The perception of space and distance of objects is much more sensitive.

This will cause the human eye to see clearly distant objects if it wants to see them clearly, and if it wants to see distant objects, the nearby objects will greatly interfere with the line of sight.

Most current HUD technologies only project a two-dimensional image at a fixed distance in front of the car. If you use animation effects to simulate fusion on this screen, it will be like the "borrowing" magic mentioned above, which is very embarrassing~ Take this AR-HUD demonstration video released by the Mercedes-Benz S-Class as an example. Although the virtual arrow will follow the It moves with the turning of the steering wheel, but when the eyes are fixed on the arrows, the road in the distance is virtual. When the focus is on the road in the distance, the arrows nearby become a flashing mess. blue.

Therefore, for AR-HUD, the distance at which virtual information is projected cannot be fixed and needs to match the position of real objects so as not to interfere with driving. Moreover, people, cars and the environment on the road are very complex, so the space in which AR-HUD projects information is not a surface, but a 3D light field with depth, and the information projected in the light field is the same as the virtual environment perceived by autonomous driving. , it is necessary to achieve precise spatial fusion and "zero delay" time fusion. Currently, there are only a few companies in the world that possess this display technology. WayRay, a Zurich-based holographic augmented reality startup, claims that its Deep Reality Display technology can project virtual information at different distances to achieve true "True AR."

If the perfect integration of virtuality and reality can be achieved, the imagination of AR-HUD will not be limited to projecting driving information. In 2021, Facebook founder Zuckerberg proposed the concept of "Metaverse" that shocked the world. The core of the concept of Metaverse is the immersive interactive experience that combines reality and reality. WayRay also took advantage of the trend and released the "Metaverse on Wheels" Holograktor concept car. . At present, WayRay has received a total of US$140 million in financing from giants such as Alibaba, Porsche, and Hyundai Motor based on the concept of "True AR", which shows the market's expectations for the future of true AR-HUD technology.

In addition to WayRay, a domestic startup company Future Black Technology (FUTURUS) also has 3D light field AR-HUD technology. FUTURUS’ exclusive light field AR-HUD system can realize the HUD screen virtual image from 4 meters to infinity. Long-range continuous zoom can truly achieve the perfect integration of virtual and real roads at the optical level. At present, FUTURUS has successfully completed the third iteration of 3D light field AR HUD real vehicle hardware, and the system has been optimized for actual road testing for three years. It is reported that the company will release its AR-HUD product at the 2022 Beijing Auto Show.

In addition to startups, technology giant Apple also uses AR-HUD as one of its secret weapons to subvert cars. In 2021, the U.S. Patent and Trademark Office granted Apple a Project Titan patent: light-field head up display. Its patent content shows that the HUD on the Apple car "can be a light field display that generates a light field output that allows the viewer to observe three-dimensional content on the head-up display. An array of light field display units and corresponding lenses can be used to convert the light field The output is directed toward the viewer. The lens directs the overlapping light field output of the display unit toward the viewer, creating a magnified, seamless light field viewing area on the car window. ”

In 2007, Apple reinvented it. The mobile phone revolutionized the keyboard phone with a clean screen. In 2012, the Model S released by Tesla also used a screen, completely subverting the interactive experience in the car cockpit. Today, 10 years later, with the development of HUD technology, a revolution in which "no screen" wins over "with screen" may have begun. Let us wait and see.