The image processor Qualcomm Spectra of the second generation will get a lot of new features
Some of the most popular features of modern smartphones are connected using built-in cameras, and it is not only about selfies and pictures of other life events, but the use of cameras in environments of augmented and virtual reality and other modern applications. The camera images are processed by a specialized processor Image Signal Processor (ISP), which plays a critical role in image quality, speed, focus and many additional features.
The company Qualcomm has announced some of the features of their Spectra ISP Qualcomm second generation with extended functionality for use in augmented and virtual reality, as well as offering improved image quality. Possible Spectra ISP is included in all smartphones based on Qualcomm Snapdragon processors.
The most interesting of the features of the Spectra of the next generation is the expansion of technology-enabled measurement of the depth (depth-sensing technology). Such technologies are known by the specialized products (the Google project Tango, the controller Microsoft Kinect and others). They allow to obtain three-dimensional maps of rented space that is used, for example, in games and applications expanded and augmented reality. Information about the depth (distance) can be used to measure the sizes of objects and tracking their position in space, which is useful in particular for tracking of user actions in virtual reality applications without the need for special motion sensors.
Signal processor Spectra ISP the next generation will support two methods of depth measurement: passive and active. The first method allows the use of only two cameras that are located next to each other, and with the help of processing differences in the two images to highlight information about the depth of objects in the scene. This method is suitable for low-cost solutions and provides the initial level of accuracy of obtaining data about the depth.
For more complex (and expensive) applications in augmented reality, Spectra ISP second generation will be able to collect information from active sensor depth using a specially prepared light infrared light and IR-sensor to build depth maps of high resolution.
Qualcomm calls these figures: obtain more than 10,000 data points of the depth with a resolution of 0.1 mm at a frequency of receipt of data at 40 frames per second or more. According to the company, they will offer the most productive and accurate depth sensors for smartphones running distance up to 3-4 meters.
The points obtained from the depth values can be used in hand tracking in virtual reality applications, tracking movement of virtual reality helmets, operating without wires and additional sensors and even recognition of faces and other objects that can be used to identify the person.
Other features of the new version of Spectra ISP can allocate hardware-accelerated denoising using data from multiple frames (multi-frame noise reduction), when an ISP gets a few frames and based on them yields a single frame, wherein deleting the data present only on some shots.
Also processor Spectra ISP second generation supports hardware acceleration of electronic image stabilization and temporal filtering with motion compensation (motion compensated temporal filtering — MCTF) to improve the quality of your videos and other opportunities that will open later.
And in order to help manufacturers to obtain quality images for processing by Spectra ISP, Qualcomm is working with partners on the program Spectra Module Program. In contrast to the conventional approach, when the smartphone manufacturer goes only to the ISP and the driver, and then he has to match it with the camera sensor by yourself, Qualcomm offers complete sets of sensors, processors, image processing ISP, and have optimized the software to work the entire kit.
In addition to the existing modules with one camera, two cameras (color and black-and-white — Qualcomm technology Clear Sight), as well as two cameras with different focal distance (optical magnification) that are already supported solutions Qualcomm, in 2017, announced support for scanning the iris of the eye and advanced capabilities to determine the depth of the scene.
Qualcomm claims that their scanner module the iris is extremely reliable, uses the IR sensor with high resolution and low power consumption, and running through the sunglasses, and that it is impossible to cheat by using three-dimensional models or two dimensional photographs, and other similar hacking techniques. And storage of biometric data uses the capabilities of protected memory ARM TrustZone Qualcomm, which will also complicate the task of attackers.
Processor Spectra ISP second generation is expected in the next flagship chip, the Snapdragon is known as 845.