In the future, autonomous drones could be used to shuttle inventory between large warehouses. A drone might fly into a semi-dark structure the size of several football fields, zipping along hundreds of identical aisles before docking at the precise spot where its shipment is needed.
Most of today’s drones would likely struggle to complete this task, since drones typically navigate outdoors using GPS, which doesn’t work in indoor environments. For indoor navigation, some drones employ computer vision or lidar, but both techniques are unreliable in dark environments or rooms with plain walls or repetitive features.
MIT researchers have introduced a new approach that enables a drone to self-localize, or determine its position, in indoor, dark, and low-visibility environments. Self-localization is a key step in autonomous navigation.
The researchers developed a system called MiFly, in which a drone uses radio frequency (RF) waves, reflected by a single tag placed in its environment, to autonomously self-localize.
Because MiFly enables self-localization with only one small tag, which could be affixed to a wall like a sticker, it would be cheaper and easier to implement than systems that require multiple tags. In addition, since the MiFly tag reflects signals sent by the drone, rather than generating its own signal, it can be operated with very low power.
Two off-the-shelf radars mounted on the drone enable it to localize in relation to the tag. Those measurements are fused with data from the drone’s onboard computer, which enables it to estimate its trajectory.
The researchers conducted hundreds of flight experiments with real drones in indoor environments, and found that MiFly consistently localized the drone to within fewer than 7 centimeters.
“As our understanding of perception and computing improves, we often forget about signals that are beyond the visible spectrum. Here, we’ve looked beyond GPS and computer vision to millimeter waves, and by doing so, we’ve opened up new capabilities for drones in indoor environments that were not possible before,” says Fadel Adib, associate professor in the Department of Electrical Engineering and Computer Science, director of the Signal Kinetics group in the MIT Media Lab, and senior author of a paper on MiFly.
Adib is joined on the paper by co-lead authors and research assistants Maisy Lam and Laura Dodds; Aline Eid, a former postdoc who is now an assistant professor at the University of Michigan; and Jimmy Hester, CTO and co-founder of Atheraxon, Inc. The research will be presented at the IEEE Conference on Computer Communications.
Backscattered signals
To enable drones to self-localize within dark, indoor environments, the researchers decided to utilize millimeter wave signals. Millimeter waves, which are commonly used in modern radars and 5G communication systems, work in the dark and can travel through everyday materials like cardboard, plastic, and interior walls.
They set out to create a system that could work with just one tag, so it would be cheaper and easier to implement in commercial environments. To ensure the device remained low power, they designed a backscatter tag that reflects millimeter wave signals sent by a drone’s onboard radar. The drone uses those reflections to self-localize.
But the drone’s radar would receive signals reflected from all over the environment, not just the tag. The researchers surmounted this challenge by employing a technique called modulation. They configured the tag to add a small frequency to the signal it scatters back to the drone.
“Now, the reflections from the surrounding environment come back at one frequency, but the reflections from the tag come back at a different frequency. This allows us to separate the responses and just look at the response from the tag,” Dodds says.
However, with just one tag and one radar, the researchers could only calculate distance measurements. They needed multiple signals to compute the drone’s location.
Rather than using more tags, they added a second radar to the drone, mounting one horizontally and one vertically. The horizontal radar has a horizontal polarization, which means it sends signals horizontally, while the vertical radar would have a vertical polarization.
They incorporated polarization into the tag’s antennas so it could isolate the separate signals sent by each radar.
“Polarized sunglasses receive a certain polarization of light and block out other polarizations. We applied the same concept to millimeter waves,” Lam explains.
In addition, they applied different modulation frequencies to the vertical and horizontal signals, further reducing interference.
Precise location estimation
This dual-polarization and dual-modulation architecture gives the drone’s spatial location. But drones also move at an angle and rotate, so to enable a drone to navigate, it must estimate its position in space with respect to six degrees of freedom — with trajectory data including pitch, yaw, and roll in addition to the usual forward/backward, left/right, and up/down.
“The drone rotation adds a lot of ambiguity to the millimeter wave estimates. This is a big problem because drones rotate quite a bit as they are flying,” Dodds says.
They overcame these challenges by utilizing the drone’s onboard inertial measurement unit, a sensor that measures acceleration as well as changes in altitude and attitude. By fusing this information with the millimeter wave measurements reflected by the tag, they enable MiFly to estimate the full six-degree-of-freedom pose of the drone in only a few milliseconds.
They tested a MiFly-equipped drone in several indoor environments, including their lab, the flight space at MIT, and the dim tunnels beneath the campus buildings. The system achieved high accuracy consistently across all environments, localizing the drone to within 7 centimeters in many experiments.
In addition, the system was nearly as accurate in situations where the tag was blocked from the drone’s view. They achieved reliable localization estimates up to 6 meters from the tag.
That distance could be extended in the future with the use of additional hardware, such as high-power amplifiers, or by improving the radar and antenna design. The researchers also plan to conduct further research by incorporating MiFly into an autonomous navigation system. This could enable a drone to decide where to fly and execute a flight path using millimeter wave technology.
“The infrastructure and localization algorithms we build up for this work are a strong foundation to go on and make them more robust to enable diverse commercial applications,” Lam says.
This research is funded, in part, by the National Science Foundation and the MIT Media Lab.
