A new system offers a more affordable and agile look at what’s around the corner by employing a computer algorithm and a simple digital camera.
What if your car not only had technology that warned you not only about objects in clear view of your vehicle—the way that cameras, radar, and laser can do now in many standard and autonomous vehicles—but also warned you about objects hidden by obstructions? Maybe it’s something a parked car is blocking, or maybe it’s just out of sight behind a building on a street corner.
This ability to see things outside your line of sight sounds like science fiction, but researchers have made strides in the last decade to bring what’s called “non-line-of-sight imaging” to reality.
Until now, they’ve had to rely on expensive and stationary equipment. But the new system could change that.
“There’s a bit of a research community around non-line-of-sight imaging,” says Vivek Goyal, an associate professor of electrical and computer engineering at Boston University.
“In a dense urban area, if you could get greater visibility around the corner, that could be significant for safety. For example, you might be able to see that there’s a child on the other side of that parked car. You can also imagine plenty of scenarios where seeing around obstructions would prove extremely useful, such as taking surveillance from the battlefield, and in search and rescue situations where you might not be able to enter an area because it’s dangerous to do so.”
In a paper in Nature, Goyal and a team of researchers say they can compute and reconstruct a scene from around a corner by capturing information from a digital photograph of a penumbra, which is the partially shaded outer region of a shadow cast by an opaque object.
“Basically, our technique allows you to see what’s around the corner by looking at a penumbra on a matte wall,” Goyal says.
Shadows on the wall
Against a matte wall, Goyal explains, light scatters equally rather than concentrating or reflecting back in one direction like a mirror. Normally, that wouldn’t give enough organized information for a computer program to translate what’s happening in a visible scene around the corner. But Goyal’s team discovered that when there is a known solid object around the corner, the partially obstructed scene creates a blurry penumbra. To the human eye, such a penumbra may not look like much. For a computer program, it’s highly informative.
By inputting the dimensions and placement of the object, the team can use their computer program to organize the light scatter and determine what the original scene looks like—all from a digital photograph of a seemingly blurry shadow on a wall.
“Based on light ray optics, we can compute and understand which subsets of the scene’s appearance influence the camera pixels,” Goyal says, and “it becomes possible to compute an image of the hidden scene.”
“When you realize how much light can be extracted from them, you just can’t look at shadows the same way again…”
For their research purposes, they created different scenes by displaying different images on an LCD monitor. But Goyal explains there’s nothing fundamental about using an LCD screen or not.
Could their approach reconstruct the image of a human being standing around the corner, for example? Goyal says there’s no conceptual barrier preventing it, but that they haven’t tried it yet. They did, however, make additional scenes by cutting out colored pieces of construction paper and pasting them on foam board to see if their system could detect the shapes and colors. Goyal says the method could interpret their “kindergarten art project.”
The most fundamental limitation is the contrast between the penumbra and the surrounding environment, Goyal explains.
“The results we present are for a relatively darkened room,” he says. When the team increased the levels of ambient light in the lab, they observed that the penumbra became harder to see and the system’s ability to precisely reconstruct the around-the-corner scene gradually became worse.
Pulling info from penumbras
Goyal says that while real-world applications for using non-line-of-sight imaging are still a ways off, the breakthrough is in the proof of concept.
“In the future, I imagine there might be some sort of hybrid method, in which the system is able to locate foreground opaque objects and factor that into the computational reconstruction of the scene,” he says.
The most exciting aspect of their findings is the discovery that so much information can be extracted from penumbras, Goyal says, which are everywhere.
“When you realize how much light can be extracted from them, you just can’t look at shadows the same way again,” he says.
The Defense Advanced Research Projects Agency (DARPA) funded the research.
Source: Boston University
