For a while now researchers at MIT and several other universities have been investigating methods for using frequencies in the WiFi bands to see through walls using a form of low power radar. The basic concept is to track and process the reflections of these signals from peoples bodies.
Recently researchers at MIT have taken this idea a step further, combining the radar results with machine learning in a project they call RF-Pose. The result is the ability to recreate and track full human post information through walls. The abstract from their paper reads:
This paper demonstrates accurate human pose estimation through walls and occlusions. We leverage the fact that wireless signals in the WiFi frequencies traverse walls and reflect off the human body. We introduce a deep neural network approach that parses such radio signals to estimate 2D poses. Since humans cannot annotate radio signals, we use state-of-the-art vision model to provide cross-modal supervision.
Specifically, during training the system uses synchronized wireless and visual inputs, extracts pose information from the visual stream, and uses it to guide the training process. Once trained, the network uses only the wireless signal for pose estimation. We show that, when tested on visible scenes, the radio-based system is almost as accurate as the vision-based system used to train it. Yet, unlike vision-based pose estimation, the radio-based system can estimate 2D poses through walls despite never trained on such scenarios.
The hope is that this technology could one day be used as a replacement for camera based computer vision. It would be a non-intrusive method for applications like gaming, monitoring the elderly for falls, motion capture during film making without the need for suits and of course for gathering data on peoples movements.
It is not mentioned in the paper, but it is likely that they are using some sort of SDR like a USRP for receiving the signals. It's possible that a lower resolution system could be set up cheaply with a HackRF and some passive radar software.
RF Pose Estimating Human Pose Behind walls using RF signals in the WiFi frequencies.Multiple people tracked with RF-Pose
Over on YouTube The Thought Emporium channel has been working on creating a "WiFi Camera" over the past few weeks. The idea is to essentially create a small radio telescope that can "see" WiFi signals, by generating a heatmap of WiFi signal strength. This is done with a directional helical 2.4 GHz antenna and motorized rotator that incrementally steps the antenna through various angles. After each movement step a HackRF and Python script is used to measure WiFi signal strength for a brief moment, and then the rotator moves onto the next angle. The helical antenna and rotator that they created are made out of PVC pipe plastic and wood, and are designed to be built by anyone with basic workshop tools like a bandsaw.
The final results show that they've been able to successfully generate heatmaps that can be overlaid on top of a photo. The areas that show higher signal strength correlate with areas on the photo where WiFi routers are placed, so the results appear to be accurate. In the future they hope to expand this idea and create a skyward pointing radio telescope for generating images of the galactic hydrogen line, and of satellites.
The Thought Emporiums' WiFi Heatmap Building Scan Results
The videos are split into three parts. The first two videos show the build process of the antennas and rotator, whilst the third video shows the final results.
DIY Radio Telescope Version 2: Wifi vision - Part 1
DIY Radio Telescope V2: Wifi Vision - Part 2
Building a Camera That Can See Wifi | Radio Telescope V2 - Part 3 SUCCESS!
To begin with Nexmon SDR you'll need a development environment set up on a Nexus 5 smartphone. Then it's a matter of downloading the dependencies, installing the Android NDK, and compiling Nexmon. IQ data can then be transmitted in code using from special system commands.
The Nexmon team have indicated on Twitter that they plan to present a paper with more information on Nexmon SDR at the MobiSys 2018 conference which will be held in June.
Thanks to Mike (ghostop14) for submitting another interesting article this time about his work with spectral fusion on the WiFi and Bluetooth bands. In the article Mike describes his new Sparrow-WiFi tool, which is a tool that allows you to visualize the WiFi and Bluetooth signal spaces all in one spectral display. The hardware consists of a WiFi and Bluetooth dongle as well as optionally an SDR like the HackRF. The software displays all data simultaneously on the same display, so you can easily tell if there is some channel clashes occurring, or if there is some other source of interference. In Addition Sparrow-WiFi also works remotely and even with a Raspberry Pi mounted on a drone.
From the article he writes:
Thinking about the 2.4 and 5 GHz bands, my biggest issues with traditional wifi tools were always that apps such as inSSIDer which are great on the Windows side didn’t have a nice polished Linux GUI equivalent so I’d have to run a Windows system or virtual machine to visualize the signal space. On the flip side, some of the great Linux-only capabilities didn’t have a nice polished integrated UI and I’d have a lot of textual data, some of which the Windows tools didn’t provide, but it was harder to visualize. Then there’s the fact that wifi tools can’t “see” Bluetooth (and vice versa), and SDR historically didn’t have enough instantaneous bandwidth to show the whole 2.4 GHz or 5 GHz spectrum at one time. And, did I mention the tools don’t integrate or talk to each other so I can’t get a “single pane of glass” perspective of all the different ways to look at the same RF space simultaneously? It would be great if I could get one single view of the most common protocols and see the actual spectrum all in one place at the same time.
Now enter the era of the Internet-of-Things, new SDR receivers, and even drones and my old wifi tools seem to have been left a bit behind. Why do I say that? I can’t “see” all of the chatter from wireless networks, Bluetooth, ZigBee, NEST devices, remotes, etc. scattered all over my wireless bands in one view. Sure, I can run 3 or 4 tools independently to find the signals and try to see what they are, but it becomes tough to get a single integrated perspective. Especially when I can’t see my RF spectrum overlaid on top of the wifi SSID’s and Bluetooth advertisements to sort out what may be related to a a signal I know about and what may be something else. Ultimately, it means that I can’t clearly explain why I have poor wifi connections in one area versus another even though I may not have overlapping channels (I know, use 5 GHz and sparrow-wifi supports that too). The reason for this is simple; current tools don’t have true spectral awareness based on the most common possibilities in one integrated solution.
Now, let’s ask even harder questions. What if I want to step up my wifi “wardriving” and start “warflying”? Or, what if I need a mobile platform that can be sent into an area on a rover? Can I bring the same spectral awareness in a small enough platform to fly for example as an under-350-gram payload complete with power, wifi, spectral scans, and even pull GPS for anything we see? And, can I interact with it remotely for real-time visibility or have it work autonomously? Okay, now you’re just asking a lot. These were all goals of a new tool I just released called “Sparrow-wifi” which is now available on GitHub (https://github.com/ghostop14/sparrow-wifi.git). Sparrow-wifi has been purpose-built from the ground up to be the next generation 2.4 GHz and 5 GHz spectral awareness and visualization tool. At its most basic, it provides a more comprehensive GUI-based replacement for tools like inSSIDer and linssid and runs specifically on Linux. In its most comprehensive use cases, Sparrow-wifi integrates wifi, software- defined radio (HackRF), advanced Bluetooth tools (traditional and Ubertooth), GPS via gpsd, and drone/rover operations using a lightweight remote agent and GPS using the Mavlink protocol in one solution.
Sparrow-Wifi Spectral Fusion. Wifi & Bluetooth dongle data + Live spectrum from a HackRF.
A full list of the possible scenarios that Sparrow-WiFi was designed for is pasted bleow.
Basic wifi SSID identification.
Wifi source hunt - Switch from normal to hunt mode to get multiple samples per second and use the telemetry windows to track a wifi source.
2.4 GHz and 5 GHz spectrum view - Overlay spectrums from Ubertooth (2.4 GHz) or HackRF (2.4 GHz and 5 GHz) in real time on top of the wifi spectrum (invaluable in poor connectivity troubleshooting when overlapping wifi doesn't seem to be the cause).
Bluetooth identification - LE advertisement listening with standard Bluetooth, full promiscuous mode in LE and classic Bluetooth with Ubertooth.
Bluetooth source hunt - Track LE advertisement sources or iBeacons with the telemetry window.
iBeacon advertisement - Advertise your own iBeacons.
Remote operations - An agent is included that provides all of the GUI functionality via a remote agent the GUI can talk to.
Drone/Rover operations - The agent can be run on systems such as a Raspberry Pi and flown on a drone (it’s made several flights on a Solo 3DR), or attached to a rover in either GUI-controlled or autonomous scan/record modes. And yes, the spectrum output works over this connection as well.
The remote agent is HTTP JSON-based so it can be integrated with other applications
Import/Export - Ability to import and export to/from CSV and JSON for easy integration and revisualization. You can also just run 'iw dev <interface> scan' and save it to a file and import that as well.
Produce Google maps when GPS coordinates are available for both discovered SSID's / Bluetooth devices or to plot the wifi telemetry over time.
The performance of WiFi networks can depend heavily on how crowded the WiFi channels are in your area. For example when your neighbours start streaming a movie over their own separate WiFi network, it can cause your own WiFi connection to slow down. This happens because generally separate WiFi networks do not collaborate with one another, and when two packets are sent on the same channel at the same time, they collide causing no packets to get through.
There are several methods that attempt to stop collisions, but none are very efficient because WiFi nodes are not synchronized to one another. If each WiFi node could be synchronized to a common reference time, then avoiding collisions is made easier.
Marcel Flores, Uri Klarman, and Aleksandar Kuzmanovic from Northwestern University have been working on this idea and have come up with a system they have termed Wi-FM which is based on FM RDS signals. Many FM radio stations transmit a digital Radio Data System (RDS) subcarrier on their broadcast frequency. This RDS signal is often used to simply display information on the radio such as the station name and current song playing.
Since each nearby WiFi node should be able to receive the same RDS signal at the exact same time, it can be used as a common synchronization signal. Then once synchronized each WiFi node can listen to the other nodes and work out what their transmit scheduling is like and then optimize their own transmit schedule.
In their prototyping they used an RTL-SDR dongle connected to a PC running GNU Radio. The GNU Radio program decodes the RDS signal and the resulting information is sent to the Linux kernel which handles the WiFi transmit schedule processing.
Researchers at the University College of London have found a way to use WiFi signals to see through walls, using a USRP software defined radio and software written in LabView. The researchers have shown that they are able to utilize local WiFi signals to detect and monitor moving objects such as people behind a wall in a similar fashion to how radar systems work. The advantage over traditional radar is that their system is completely passive, requiring no transmitter, other than the already ubiquitous WiFi signal.
In a demonstration the researchers showed how they were able to not only detect the presence of a person behind a wall, but also detect small hand gestures that were made.
Detecting body gestures from WiFi signals in LabView.
It appears the researchers are patenting their work and are looking to market their technology towards military and security surveillance operators as well as towards other applications such as traffic monitoring and the monitoring of children and the elderly.
We aren’t sure what type of radio accuracy is required for a system such as this, but it may be possible that SDR’s that cost less than the USRP may also work, assuming the software technology can ever be replicated/licensed.
A proposed application of the technology: Allowing police to see through walls in a hostage situation.
