Sensor fusion is also known as multi-sensor data fusion. It appears to be a straightforward process. Two or more sensors are preferable than a single sensor. When you combine them, you have sensor fusion! Well, the software and algorithms that enable sensor fusion will have you reconsidering that “simple” term in no time.
Definition of Sensor Fusion
The capacity to combine data from many radars, lidars, and cameras to generate a single model or image of the world surrounding a vehicle is known as sensor fusion. Because the strengths of the many sensors are balancing, the final model is more accurate. Sensor fusion information may then be used by vehicle systems to enable more intelligent behaviors.
Each sensor type, or “modality,” has its own set of advantages and disadvantages. Radars are excellent at estimating distance and speed, especially in adverse weather. But they can’t read street signs or “see” the color of a stoplight. Cameras are excellent at interpreting signs and categorizing things like people, bikers, and other cars. Dirt, light, rain, snow, or darkness, on the other hand, may easily blind them. Lidars can detect things precisely, but they lack the range and price of cameras and radar.
How Does Sensor Fusion Work?
Sensor fusion combines data from all of these sensor types and applies software algorithms to create the most comprehensive and accurate environmental model possible. Through a technique calling as internal and external sensor fusion, it may also correlate data from inside the cabin.
A vehicle might also employ sensor fusion to combine data from numerous sensors of the same sort, such as radar. Taking use of slightly overlapping areas of view, this increases perception. More than one sensor will identify things at the same time. Because numerous radars are monitoring the area surrounding a vehicle. Detections from those many sensors may be overlapping or merging using global 360° perception software, boosting the detection probability and reliability of things around the vehicle and producing a more accurate and trustworthy picture of the environment.
Low-level sensor fusion
Of course, the more sensors on a vehicle, the more difficult fusion becomes, but the more opportunities for performance improvement present. Aptiv employs a technology known as low-level sensor fusion to reap these benefits.
Previously, the computing power requiring to evaluate sensor data in order to discern and track objects was packing with cameras or radars. The processing capacity is concentrating into a more powerful active safety domain controller with Aptiv’s Satellite Architecture method, allowing low-level sensor data to be gathering from each sensor and fusing in the domain controller.
Advantage of low-level Sensor Fusion
Sensors that have their processing moving to a domain controller take up less space and weigh less – up to 30% less. A camera’s footprint is lowering from the size of a deck of cards to the size of a pack of chewing gum for comparison. OEMs have more alternatives in car packaging by keeping sensors as compact as feasible.
Increasing data exchange is another advantage. Smart sensors, unlike traditional systems, evaluate environmental inputs separately. And it implies that any judgments made basing on the information are only as good as what each sensor can observe. However, with Satellite Architecture, where all data from sensors is sharing centrally, active safety applications in the domain controller have additional opportunities to exploit it. Aptiv can even utilize artificial intelligence (AI) methods to extract relevant data from data that would otherwise be thrown away. The correct AI can learn from it, which aids us in solving difficult corner situations that our clients confront.
Reducing latency is a third advantage of low-level sensor fusion. It is not necessary for the domain controller to wait for the sensor to process data before acting on it. This can aid with speed in instances where fractions of a second matter.
Better judgments are made when there is more data. Vehicles may become smarter and quicker by adopting a vehicle design that allows for a large number of sensors and then synthesizes the data through sensor fusion.
Interior and Exterior Sensor Fusion
Every radar, camera, lidar — every sensor in a vehicle — doesn’t simply add to the safety capabilities; it doubles them in the domain of active safety. With additional inputs and data points to deal with, the correct software can combine the data to create judgments that go much beyond what sensor fusion has historically been thought of as.
Sensor fusion, in most cases, refers to the combining electronic components of data inputs from two or more external sensors in order to gain a better read on what impediments may be present surrounding a vehicle. For example, we might combine the outputs of two radars to characterize an item that emerges on the outside of their respective fields of vision. Alternatively, we might combine the data from a camera and a radar to take use of both modalities’ strengths and more correctly determine what’s around the car.
External Radars
We can see that your eyes are gazing down at the radio instead than out at the road because of interior sensing. But the external radars spot the automobile moving into your lane. We can instantly alert you that something is happening outside the car that demands your attention. And we can even lead you to the location using visual and audible signals.
Much more proactive safety solutions are now feasible because to the integration of inside and outside sensors. Consider the following scenario: you’re waiting to make a right-hand turn at an intersection. You’re scanning the left side of the road for a space in the traffic so you can make a swift right turn. Meanwhile, a pedestrian approaches from the right and begins to cross in front of you.
Inner Sensor
Although the inner sensor detects your gaze, one of the outer radars detects the pedestrian. Because you didn’t look over to observe the pedestrian. The system alerts you that he or she is present – before it becomes an emergency.
Conclusion
In conclusion, sensor fusion tries to transcend the limits of individual sensors by combining data from several sensors. And they provide more accurate and less uncertain information. This more reliable data may subsequently be utilizing to make judgments or carry out specific activities. The magic behind the curtain appears to be simple. So a microcontroller utilizes software algorithms to aggregate or fuse data from numerous sensors to provide a more comprehensive picture of the process or scenario in question. The concept is that a better knowledge of the process or issue leads to more and/or deeper insights. They can lead to new, more intelligent, and more accurate solutions.
Next Post: What is an SoC and How does it Work
