Computer vision researchers report using special light sources and sensors to see around corners or through gauzy filters, letting them reconstruct the shapes of unseen objects.
The researchers say this technique enables them to reconstruct images in great detail, including the relief of George Washington’s profile on a US quarter.
Ioannis Gkioulekas, an assistant professor in Carnegie Mellon University’s Robotics Institute, says this is the first time researchers have been able to compute millimeter- and micrometer-scale shapes of curved objects, providing an important new component to a larger suite of non-line-of-sight (NLOS) imaging techniques now being developed by computer vision researchers.
“It is exciting to see the quality of reconstructions of hidden objects get closer to the scans we’re used to seeing for objects that are in the line of sight,” says Srinivasa Narasimhan, a professor in the Robotics Institute. “Thus far, we can achieve this level of detail for only relatively small areas, but this capability will complement other NLOS techniques.”
Bouncing light
Most of what people see—and what cameras detect—comes from light that reflects off an object and bounces directly to the eye or the lens. But light also reflects off the objects in other directions, bouncing off walls and objects. A faint bit of this scattered light ultimately might reach the eye or the lens, but is washed out by more direct, powerful light sources. NLOS techniques try to extract information from scattered light—naturally occurring or otherwise—and produce images of scenes, objects, or parts of objects not otherwise visible.
“Other NLOS researchers have already demonstrated NLOS imaging systems that can understand room-size scenes, or even extract information using only naturally occurring light,” Gkioulekas says. “We’re doing something that’s complementary to those approaches—enabling NLOS systems to capture fine detail over a small area.”
Like lidar
In this case, the researchers used an ultrafast laser to bounce light off a wall to illuminate a hidden object. By knowing when the laser fired pulses of light, the researchers could calculate the time the light took to reflect off the object, bounce off the wall on its return trip, and reach a sensor.
“This time-of-flight technique is similar to that of the lidars often used by self-driving cars to build a 3D map of the car’s surroundings,” says Shumian Xin, a PhD student in robotics.
Previous attempts to use these time-of-flight calculations to reconstruct an image of the object have depended on the brightness of the reflections off it. But in this study, Gkioulekas says the researchers developed a new method based purely on the geometry of the object, which in turn enabled them to create an algorithm for measuring its curvature.
The researchers used an imaging system that is effectively a lidar capable of sensing single particles of light to test the technique on objects such as a plastic jug, a glass bowl, a plastic bowl, and a ball bearing. They also combined this technique with an imaging method called optical coherence tomography to reconstruct the images of US quarters.
In addition to seeing around corners, the technique proved effective in seeing through diffusing filters, such as thick paper.
So far, researchers have demonstrated the technique only at short distances—a meter at most. But the researchers speculate that their technique, based on geometric measurements of objects, might be combined with other, complementary approaches to improve NLOS imaging. It might also be employed in other applications, such as seismic imaging and acoustic and ultrasound imaging.
The research team includes others from Carnegie Mellon, University College London, and the University of Toronto. They are part of a larger collaborative team that is developing a suite of complementary techniques for NLOS imaging.
Researchers presented the work at the 2019 Conference on Computer Vision and Pattern Recognition (CVPR2019) in Long Beach, California.
DARPA, the National Science Foundation, the Office of Naval Research, and the Natural Sciences and Engineering Research Council of Canada supported this research.
Source: Carnegie Mellon University
