A new system uses video of a person walking to identify that person through walls using two WiFi transceivers.
This novel video-WiFi cross-modal gait-based person identification system, which they refer to as XModal-ID (pronounced Cross-Modal-ID), could have a variety of applications, from surveillance and security to smart homes.
For instance, consider a scenario in which law enforcement has a video footage of a robbery. They suspect that the robber is hiding inside a house. The new technology could determine if the person inside is the same person in the video.
“Our proposed approach makes it possible to determine if the person behind the wall is the same as the one in video footage, using only a pair of off-the-shelf WiFi transceivers outside,” says Mostofi. “This approach utilizes only received power measurements of a WiFi link. It does not need any prior WiFi or video training data of the person to be identified. It also does not need any knowledge of the operation area.”
In the team’s experiments, one WiFi transmitter and one WiFi receiver are behind walls, outside a room where a person is walking. The transmitter sends a wireless signal whose power the receiver measures. Then, given video footage of a person from another area—and by using only such received wireless power measurements—the receiver can determine whether the person behind the wall is the same person seen in the video footage.
Our unique moves
This innovation builds on previous work in the Mostofi Lab, which has pioneered sensing with everyday radio frequency signals such as WiFi since 2009.
“However, identifying a person through walls, from candidate video footage, is a considerably challenging problem,” says Mostofi. Her lab’s success in this endeavor is due to the new proposed methodology they developed.
“The way each one of us moves is unique. But how do we properly capture and compare the gait information content of the video and WiFi signals to establish if they belong to the same person?”
The researchers have proposed a new way that can translate the video gait content to the wireless domain to identify people through walls.
“Our approach is multi-disciplinary, drawing from areas of both wireless communications and vision,” says Chitra Karanam, one of three PhD students on the project. Given some video footage, the team first utilized a human mesh recovery algorithm to extract the 3D mesh describing the outer surface of the human body as a function of time. They then used Born electromagnetic wave approximation to simulate the RF signal that would have been generated if this person was walking in a WiFi area.
Next they employed their time-frequency processing approach to extract key gait features from both the real WiFi signal (that was measured behind the wall) and the video-based simulated signal. The system then compares the two signals to determine if the person in the WiFi area is the same person in the video.
Through the walls
The researchers’ processing pipeline involves a series of mathematical functions, including Short-time Fourier transform and Hermite functions, in order to get the spectrogram of the received signal.
“A spectrogram carries the frequency-time content of the signal, which implicitly carries the gait information of the person,” explains Belal Korany, another PhD student involved in the effort.
Several important gait features are then extracted from both spectrograms and properly compared to declare if the person in the video is behind the wall.
“We have tested this technology extensively on our campus,” says Herbert Cai, the third PhD student on the project. The lab has tested their new technology on 1,488 WiFi-video pairs, which researchers drew from a pool of 8 people, and in 3 different behind-wall areas, and achieved an overall accuracy of 84% in correctly identifying the person behind the wall.
The proposed methodology and experimental results will appear at the 25th International Conference on Mobile Computing and Networking (MobiCom). Funding for the project came from a pair of grants from the National Science Foundation that focus on through-wall imaging and occupancy assessment.
Source: UC Santa Barbara
