Shortly in Finnish: Mitä etuja millimetriaaltotutka tarjoaa havainnointiongelmissa, joissa näkyvyydessä on puutteita tai yksityisyydensuoja on ensisijaista? Lue tästä, miten me Emblicalla olemme ratkaisseet mmWave-tekniikan avulla ongelmia, joissa kameraa hyödyntävät konenäköratkaisut eivät ole olleet mahdollisia.
For the past several years, automation and AI have had great levels of interest in almost any industry from car R&D to agriculture 1 2. Computer Vision (CV) in particular is a domain that continues to find countless applications via Machine Learning. However, these solutions largely rely on Camera-based computer vision—perhaps as it is the most intuitive kind of vision to us humans. At Emblica, we're comfortable using also less common tools and adopting a technology that is not common knowledge even in the domain of Machine Learning specialists we operate in. This can provide our customers unique solutions to problems otherwise deemed too difficult to solve. In this blog, we focus on one alternative to Camera-based Computer Vision, the millimeter-wave radar.
Limitations of Camera-based Computer Vision
As mentioned, camera-based systems are not always suitable for CV applications. Here are some reasons why:
-
Privacy concerns: Despite the availability of technology to blur faces, privacy remains a concern. There is always the risk of malicious third-party access to raw video data, even if it is anonymized in post-processing.
-
Environmental issues: Cameras are susceptible to weather conditions, such as fog or dirt on the lens. This makes designing camera-based systems more complicated.
-
Hardware requirements: The specific hardware requirements for camera-based systems can be challenging.
Alternatives to Cameras
In light of the challenges posed by camera-based computer vision solutions, it is important to consider alternative technologies that can address these issues. One such technology is radars, which offer several advantages over cameras. For instance, radar signals cannot be used to identify facial features, thus addressing privacy concerns. Additionally, radars can penetrate through sparse matter, allowing for clear radar images even in low-visibility conditions like dust, rain, or snowfall. Furthermore, radars do not have a camera lens, eliminating the risk of obstructions in the field of view. These features make radars a promising technology for many special Computer Vision applications where cameras are inadequate.
One well-known technology—in many ways analogous to radars—are LiDAR-based systems. While Devices utilizing the LiDAR-technology have found a plenty of use in many industrial applications like self-driving cars 3. For those interested to read more about LiDAR-technology, here is our blog post on the topic. However, a downside of the technology is these devices are very expensive. This makes them impractical for many applications where the system must be affordable for mass production.
A millimeter-wave radar (mmWave radar) is a small, relatively cheap (available at prices well under 200 euro at the time of publication) radar that operates on a very precise millimeter-wave signal. It is precise enough that it is possible to discover the position and gait of a person even through sparse matter—such as obstructions on the lens—without discovering the facial features.
What is a Millimeter-Wave Radar?
A mmWave radar is a type of radar that emits short-wavelength electronic waves known as millimeter-waves (mmWaves), that objects in their path then reflect back 4. The radar captures these reflected signals, and the underlying software of the system can usually calculate properties such as the range, relative radial velocity, and angle of arrival of targets within the signal. These are physical properties traditional to a radar-system, usually obtained via signal processing. This is in contrast to a camera, whose signal only reveals the 2D position.
An interesting property of the mmWave-signal in particular is that its wavelength in the electromagnetic spectrum is short. This has two advantages: Firstly, the radar system can be relatively small in size. Secondly, the accuracy of the signal is high. For example, according to Texas Instruments, a mmWave radar operating at 76–81 GHz can detect movements as small as a fraction of a millimeter 4.
Because the domain is very different to camera-based computer vision, discovering important information from signals like mmWave-signal requires sophisticated technology and specialized expertise. Machine Learning tools can be a great help here.
Moreover, many off-the-shelf systems of mmWave radars come with firmware for transforming the radar signal into a three-dimensional point cloud. Once objects are observed in the vicinity, their position and relative radial velocity can be seen in the point cloud data output by the system. This is also where Machine Learning and AI come into picture. Because the domain is very different to camera-based computer vision, discovering important information from signals like mmWave-signal requires sophisticated technology and specialized expertise. Machine Learning tools can be a great help here.
Millimeter-Wave Radar Applications
Millimeter-wave radar has numerous potential applications. One such use case is the detection and counting of humans within the radar's range. For instance, smart doors or elevators that can distinguish between someone simply walking by and someone waiting for the door to open. Another potential application is detecting falls among elderly residents in nursing homes, which can be crucial for their well-being and health. While difficult problems for classical algorithms, such tasks can be automated with modern Machine Learning tools, thanks to their capability to deal with large masses of data, without compromising the individuals' privacy.
While difficult problems for classical algorithms, such tasks can be automated with modern Machine Learning tools, thanks to their capability to deal with large masses of data, without compromising the individuals' privacy.
Even outside cars, autonomous vehicles and machines are prime example of environment where radar can be used and where some of our customers are developing new solutions.
Object detection solutions with the mmWave radar are more common than one might think. The radar is used by several car manufacturers (e.g. Tesla, Mercedes-Benz, and Audi 5) in sensor fusion combined with other radar types like LiDARs to improve the capabilities of the cars' autonomous driving systems to make decisions.
Even outside cars, autonomous vehicles and machines are prime example of environment where radar can be used and where some of our customers are developing new solutions.
mmWave radars have been used to measure heart beats and breathing remotely.
Advantages in comparison to using just the more well-known LiDARs include improved long-range object detection, lower price, and more accurate detection of dynamic targets. In comparison to vision sensors, the object detection performance of mmWave radars is also affected less by extreme weather.
The possibilities do not end with object detection, mmWave radars have been used to measure heart beats and breathing remotely and with sophisticated algorithms object identification from the signal is also viable. Maybe we would like to know how many shopping mall customers prefer to use shopping carts or hand-held baskets? The difference in the objects can be seen in the mmWave-signal, and extracted using Machine Learning tools.
Human Identification and Tracking in Indoor Environments
At Emblica, we have investigated the viability of the mmWave radar in the problem domain of human detection and tracking in an indoor space. In particular, we investigated the Emblica office and the public transport. These are good examples of environments where the mmWave radar has an advantage: (1) The spaces are in the public, so anonymity has a high priority. (2) Public surfaces are prone to getting dirty and degraded, either due to natural wear and tear or due to harassment.
This task can be reduced to a problem of tracking humans in an indoor space. To this end, Emblica engineered the MMWR-Tracker—an end-to-end system for human detection and tracking in an indoor environment.
The system uses the U-Net architecture 6 —A Deep Neural Network for Image Segmentation applications—to first segment the mmWave-signal from the mmWave radar system into a segmentation map where signal noise is filtered away and humans detected by the signal remain. These segmentation maps are passed to a tracking module, that utilizes a Kalman filter and the Hungarian Method—methods commonly found in object tracking solutions in the domain—to track the humans detected.
Once installed in an edge device backend, the MMWR-Tracker works in real-time, monitoring the environment and relaying the spatial information of detected objects to the cloud.
Tracking humans is just one possible application of the radar technology. Other possibilities include detecting whether a human or a pet is moving in its field of view—think of smart lighting or security systems.
Conclusion
In conclusion, while many computer vision solutions rely on cameras, the mmWave radar offers an alternative solution that is especially useful in scenarios where visibility and privacy are limited. Although not as well-known as other computer vision tools, such as stereo cameras or LiDAR systems, the mmWave radar has unique properties that make it a valuable consideration for specific use cases. Emblica has explored the potential of this technology to address problems that cannot be solved with camera-based computer vision solutions.
Emblica is not your average data team. We build customized solutions for collecting, processing, and utilizing data for all sectors, especially at the R&D interface. Whether our target is a factory line, an online store, or a bus, you can find us busy at work, hands in the clay - at least at our office in Helsinki.