Surveillance Flying robots equipped with 3D gear: better surveillance on the cheap

Sensor system would prevent mini-drone to fly in swarms // Source: ning.com

They can be deployed as additional surveillance resources during major events, or as high-resolution 3D street imaging systems. Intelligent swarms of aerial drones are a useful tool for police, first responders, crisis managers, and urban planners. It is thus good news that special 3D sensors developed by Fraunhofer researchers allow for flawless aerobatics and prevent collisions.

As with a well-rehearsed formation team, a flock of flying robots rises slowly into the air with a loud buzzing noise. The two dozen robots perform an intricate dance in the sky above a stadium full of soccer fans. Rowdy hooligans have stormed the field and set off flares.

Fights are breaking out all over, smoke is hindering visibility. Only the swarm of flying drones can maintain an overview of the situation. These UAVs are a kind of mini-helicopter, with a wingspan of around two meters. They have a propeller on each of their two variable-geometry side wings, which gives them rapid and precise maneuverability. In operation over the playing field, their cameras and sensors capture images and data, and transmit them to the control center. Where are the most seriously injured people? What is the best way to separate the rival gangs in the stands? The information provided by the drones allows security personnel to make decisions more quickly, all the while the robots go about their surveillance business above the arena autonomously, and without ever colliding with each other, or with any other obstacles.

A Fraunhofer IMS release reports that a CMOS sensor developed by researchers at the Fraunhofer Institute for Microelectronic Circuits and Systems IMS in Duisburg, Germany, is at the heart of the anti-collision technology. “The sensor can measure three-dimensional distances very efficiently,” says Werner Brockherde, head of the development department. Just as in a black and white camera, every pixel on the sensor is given a gray value.

“But on top of that,” he explains, “each pixel is also assigned a distance value.” This enables the drones accurately to determine their position in relation to other objects around them.

The distance sensor developed by the IMS offers significant advantages over radar, which measures distances using reflected echoes. “The sensor has a much higher local resolution,” says Brockherde.

“Given the near-field operating conditions, radar images would be far too coarse.” The flying robots are capable of identifying even small objects measuring 20 by 15 centimeters at ranges of up to 7.5 meters. Moreover, this distance information is then transmitted at a rate of twelve images per second.

Even when there is interfering light, for example, when a drone is flying directly into the sun, the sensor will deliver accurate images. It operates according to the time-of-flight (TOF) process, whereby light sources emit short pulses which are reflected by objects and bounced back to the sensor. In order to prevent over-bright ambient light from masking the signal, the electronic shutter only opens for a few nanoseconds. In addition, the sensor also takes differential measurements, in which the first image is captured using ambient light only, a second is taken using the light pulse as well, and the difference between the two determines the required output signal.  “All of this happens in real time,” adds Brockherde.

The release notes that the 3D distance sensors are built into cameras manufactured by TriDiCam, a spin-off company of Fraunhofer IMS. Jochen Noell, TriDiCam’s managing director, admits: “This research project has presented us with new challenges as regards ambient operating conditions and the safety of the sensor technology.” The work falls under the AVIGLE project, one of the winners of the Hightech.NRW adcanced technology competition which receives funding from both the State of North Rhine-Westphalia and the EU. The IMS engineers will be presenting their sensor technology at the Fraunhofer CMOS Imaging Workshop in Duisburg on 12 and 13 June this year.

Conducting intelligent aerial surveillance of major events is not the only intended use for flying robots. They could also be of benefit to disaster relief workers, first responders, and urban planners, who could use them to produce detailed 3D models of streets or to inspect roofs in order to establish their suitability for solar installations. Whether deployed to create virtual maps of difficult-to-access areas, to monitor construction sites or to measure contamination at nuclear power plants, these mini UAVs could be used in a wide range of applications, obviating the need for expensive aerial photography or satellite imaging.