Match the surgery with GPS! Peking Union Medical College and Tencent released the domestic autonomous surgery navigation system.

The system consists of navigation tablet computer, navigation bar, depth camera and other equipment. Small and portable, does not occupy the operating room space. This is a neurosurgical navigation system, which can be located at the bedside. It has the characteristics of high precision, portability and low cost. Just like providing a GPS for clinicians' surgery, it helps doctors accurately locate the lesions hidden in brain tissue.

At present, the product has carried out more than 30 clinical trials in the neurosurgery department of Peking Union Medical College Hospital, involving cerebral hemorrhage, pituitary adenoma, glioma, meningioma, craniopharyngioma and other diseases.

Professor Wang Renzhi, director of neurosurgery and doctoral supervisor of Peking Union Medical College, said: "Neurosurgeons not only require high accuracy in locating lesions during operation, but also need to know the relationship between lesions and peripheral blood vessels and nerves to avoid damaging these important tissues during operation. In the past, we only relied on experience to re-integrate the two-dimensional CT or magnetic resonance images of patients into the brain to form three-dimensional images. In fact, this learning process is very long and requires a lot of long-term experience accumulation. The surgical navigation developed by us in cooperation with Tencent is simple, convenient and easy to use, which can help clinicians solve many practical problems. "

It is understood that the utilization rate of traditional surgical navigation in China is very low, because: the core technology and components of traditional surgical navigation system rely on imports, and this technology is difficult to really benefit primary hospitals, costing millions; The traditional surgical navigation equipment is bulky, with high learning and operation threshold and long use time, so it is difficult to be really applied in practical scenes such as emergency.

In order to solve these problems fundamentally, the R&D team has made a subversive and innovative design for surgical navigation for three years, which not only greatly reduced the cost, but also saved the preparation time. Compared with foreign products, the accuracy is equivalent, and the localization replacement of surgical navigation system is completed. On the premise of high precision, the new navigation reduces its cost to one tenth of the traditional surgical navigation products, filling the gap in the domestic market.

Based on the cutting-edge algorithm ability of Tencent AI Lab and the clinical accumulation of neurosurgery in Peking Union Medical College Hospital for many years, the domestic surgical navigation system can assist doctors to improve the efficiency of diagnosis and treatment before and during surgery.

Before surgery, based on artificial intelligence and imaging technology, the surgical navigation system can automatically segment, reconstruct and visualize images such as magnetic vibration and ct, and through the algorithmic ability of Tencent AI Lab, a customized three-dimensional "transparent brain" can be generated with one click, which is convenient for doctors to make surgical plans and explain them to patients and their families.

The whole system integrates optical positioning and SLAM tracking. Through optical positioning, the surgical navigation system can track the surgical instruments in real time. Through the real-time localization technology (SLAM), the system can observe the position of the lesion from multiple angles without a head frame and a reference frame.

During the operation, SLAM tracking can realize augmented reality navigation without headgear and reference frame. Taking a glioma patient as an example, when the doctor moves the camera, the system can display the position relationship of the three-dimensional structure of the patient's skull relative to the scalp in real time, and assist the doctor to make an incision quickly.

With the help of depth camera, combined with deep learning and vision algorithm, the system can achieve high-precision one-click registration between virtual space and real space, and match the generated 3D "transparent brain" 1: 1 with the actual situation of patients for intraoperative navigation.

The data show that the registration accuracy can reach sub-millimeter level. The improvement of algorithm ability can shorten the registration time of traditional surgical navigation from nearly half an hour to a few seconds, and seize valuable treatment opportunities for patients.

In practical application scenarios, the system can also observe the brain from multiple perspectives according to the needs of doctors, and remove the visual blind spots through cooperation with AR glasses.

In addition to neurosurgery, surgical navigation system can also be used in otolaryngology and emergency department in the future. The system can also be used in classroom teaching to improve the ability of primary medical service.

Tencent has a number of layouts in the fields of medical care and life sciences. When the epidemic broke out in 2020, Tencent AI Lab and Academician Zhong Nanshan jointly published a research result on predicting the critical probability of patients with coronavirus pneumonia-19 AI, which was published in the sub-journal Nature. At present, Tencent is combining computer vision, machine learning, deep learning and other technologies with medicine to assist doctors in disease screening, diagnosis and treatment, improve the diagnostic accuracy of clinicians and save more patients' lives.