Research for revolutionary technology for seeing through walls

Science fiction writers can dream up any advanced technology without having to build anything. Many of them “invented” fantastic devices before they actually came onto the market and into homes and cities. 

How about an algorithm that computes highly accurate, full-color, three-dimensional reconstructions of areas behind obstacles – a concept that can not only help prevent car crashes but also help IDF combat forces in the underground “city” populated by Hamas terrorists in Gaza, law enforcement experts in hostage situations and search-and-rescue efforts. 

In fact, researchers at the University of South Florida have actually accomplished such a technology. While the algorithm is still in the development phase, it is available for other researchers to test and reproduce in their own space.

After a recent car crash, computer science, and engineering Prof. John Murray-Bruce wished he could have seen the other car coming. The crash reaffirmed the importance of his mission to create a technology that could see around obstacles and ultimately expand one’s line of vision.

Using a single photograph, Murray-Bruce and his doctoral student Robinson Czajkowski created an algorithm that computes highly accurate, full-color, three-dimensional reconstructions of areas behind obstacles.

“We’re turning ordinary surfaces into mirrors to reveal regions, objects, and rooms that are outside our line of vision,” Murray-Bruce said. “We live in a 3D world, so obtaining a more complete 3D picture of a scenario can be critical in a number of situations and applications.”

In 10 to 20 years, technology could be adopted by law enforcement

Czajkowski and Murray-Bruce expect it will be 10 to 20 years before the technology is robust enough to be adopted by law enforcement and car manufacturers, but maybe Israeli engineers could put their heads together and develop their own technology based on this idea long before. Now, the University of South Florida team plans to continue their research to improve the technology’s speed and accuracy further to expand its applications in the future, including self-driving cars to improve their safety and situational awareness.

The team members have just published in the prestigious journal Nature Communications under the title “Two-edge-resolved 3D, non-line-of-sight imaging with an ordinary camera,” the first-of-its-kind technique to successfully reconstruct a hidden scene in 3D using an ordinary digital camera. The algorithm, which works by using information from the photo of faint shadows cast on nearby surfaces to create a high-quality reconstruction of the scene, could have broad applications.

“These shadows are all around us,” Czajkowski said. “The fact we can’t see them with our naked eye doesn’t mean they’re not there.” The idea of seeing around obstacles has been a topic of science-fiction movies and books for decades. Murray-Bruce commented that their research takes significant strides in bringing that concept to life.

Although in the past, significant strides have been made, notably by the class of works based on the corner camera that exploit vertical edges such as those occurring in corridors, doorways, or at the boundaries of buildings in urban environments to form reconstructions of a hidden scene, intensity-based passive methods are limited to one-dimensional and two-dimensional reconstructions. The most successful demonstrations of 3D imaging of hidden scenes all required specialized, expensive equipment.

“Our work achieves a similar result using far less,” Czajkowski said. “You don’t need to spend a million dollars on equipment for this anymore.”

“In just over a decade since the idea of seeing around corners emerged, there has been remarkable progress, and there is accelerating interest and research activity in the area,” Murray-Bruce said. “This increased activity, along with access to better, more sensitive cameras and faster computing power, form the basis for my optimism on how soon this technology will become practical for a wide range of scenarios.”