The first primitive unmanned aerial vehicles (UAVs) emerged in 1849, when Austrian forces, in an attempt to quash a rebellion in Venice, used hot air balloons launched from a ship 4 miles away to bomb the city. Although that particular plan didn’t prove to be an effective military maneuver—some of the balloons apparently drifted back over Austrian lines—the usefulness of UAVs was immediately clear: They could gather intel or conduct an operation without risking the life of a human operator.
But UAVs—which can range in size from a small aircraft to a flyer that fits in the palm of your hand—are also lauded for their efficiency and cost benefits, and in the last few years the commercial market has, well, taken off. They've been used successfully in a number of applications, from monitoring crop health to inspecting property damage. Consulting firm McKinsey & Company estimated that commercial drone activity was worth $1 billion in 2017, compared to $40 million just five years before.
Still, much work on these flying computers remains to be done, and that's where UTA comes in. From its numerous National Science Foundation (NSF) grants to its certificate programs training the next generation of experts, the University continues to make its mark in UAV research, helping this dynamic industry find its wings.
Improving Data Collection
In April 2018, Anand Puppala, associate dean for research in the College of Engineering, wrapped up an 18-month research project with the Texas Department of Transportation (TxDOT) that investigated the use of UAVs for infrastructure monitoring. He and his team conducted surveys of roads, a bridge, a railroad corridor, and multiple materials stockpiles, and worked with TxDOT to vet their flight operations manual. The researchers found that UAVs provide data more quickly, safely, and cheaply than traditional monitoring methods.
Under Dr. Puppala's guidance, Cody Lundberg, a research engineer at the UTA Research Institute (UTARI) and an FAA-certified UAV pilot, along with postdoctoral scholar Surya Sarat Chandra Congress traveled around the state gathering high-definition infrastructure images. In Sherman, Texas, for example, the duo conducted two 10-minute UAV flights along a stretch of highway, using the attached digital camera to capture data about pavement conditions such as roughness, rutting, and cracking. Back at UTA, Dr. Congress geotagged the images and created a dense point cloud from the data. With these virtual models, the team could then analyze different pavement performance characteristics in fine detail using commercially available image analysis software and UTA-developed algorithms.
When Congress and Lundberg conducted their flights, warning signs were posted, but all lanes remained open, and the pilot and crew were stationed at a safe distance from the road. In addition to increasing safety, using UAVs can save time because the drone gathers all of the data about each attribute simultaneously and in the same way—there's no need to take separate measurements for each characteristic as with traditional methods.
Congress believes that with the current technology, UAVs can serve as a complement to those traditional methods by helping transportation personnel zero in on problematic areas.
College of Engineering Associate Dean Anand Puppala prepares a UAV for testing at the UTA Research Institute.
"You can collect the data of thousands of miles with UAVs to find the critical areas and then conduct the costly traditional surveys on those specific stretches."
Puppala echoes that sentiment. "The accuracy [of UAV-provided data] won't be as good as the expensive laser-based profilers, but if I can get close, not too far from the real values, I think people may consider this as a tool to use."
An implementation phase of the project began in September. The researchers are spending six months developing training materials, and Congress has traveled to nine major TxDOT districts to conduct training sessions with engineers, surveyors, and administrators on using UAVs in the field for data collection and on flight operation protocols. The team also demonstrated the data collection abilities of UAVs in different applications—intersection, building, and planimetric mapping; pavement forensics; stockpile volumetric studies; and bridge and communication tower surveying.
This work with TxDOT has served as a springboard for other projects. With aid from a yearlong Rapid Response Research grant from the NSF, Puppala and collaborators from Louisiana State University used UAVs and smartphones to capture images of debris piles from 2017's Hurricane Harvey at eight sites in Beaumont, Texas. Once back at UTA, Congress used photogrammetry—the science of measuring distances from two or more photographs—to create highly accurate 3D models. The purpose was to develop a step-by-step methodology that could better estimate the volumes of piles of trash, which in turn could help municipalities improve future cleanup efforts and allow them to budget appropriately.
"Our goal was to really help them understand from an engineering point of view how to estimate the amount of volume so they could get a better handle on how long the cleanup process takes and how many truckloads they'll need," Puppala says. "All of those things they can plan."
He and his team are also building on their work in a new project with the Harris County Toll Road Authority, which oversees toll roads in the Houston area. The researchers will use UAVs to assess the conditions of bridges, roads, and existing signage.
"Infrastructure dollars are very limited," Puppala says, "and the UAVs quickly and safely provide important data that helps agencies identify where they want to invest their dollars for maximum value."
Enabling UAV Teamwork
Despite the benefits UAVs bring, the technology does present challenges. For example, it can be difficult for a single vehicle to perform a complex mission because, due to its small size, the drone may not be able to carry all the needed sensors. Using multiple UAVs brings its own issues, however: If the mission has a series of tasks that must occur in a certain order, it may not be best to launch the UAVs together because they have relatively limited flying times. Ideally, the UAVs need to be able to communicate with each other to figure out the optimal sequence for collecting data.
"Though each of them is highly optimal, when you put them together and you restrict communication between certain channels, the performance of the overall team may not be as optimal," says Kamesh Subbarao, associate professor in the Mechanical and Aerospace Engineering Department. "If it is not optimal, then what does it depend upon and how can we mitigate those circumstances?"
Dr. Subbarao, who is also the director of UTA's Aerospace Systems Laboratory, is using a four-year, $795,427 Basic Research Challenge grant from the Office of Naval Research (ONR) to answer those questions. He'll quantify how teams of UAVs perform in coordination by examining the sensitivity of individual vehicles to certain variables and then build algorithms based on those metrics that will allow operators to effectively predict how these grouped vehicles will work together.
Associate Professor Kamesh Subbarao
One metric, for example, examines how multiple vehicles perform when they are being directed to a GPS location and need to determine which vehicles are the closest. A second metric looks at how they pass information when only some of them know where to go. And a third examines how closely they can fly without colliding and what conditions might affect that answer.
"What complicates the whole thing is the communication dynamic," Subbarao says, noting that those networks can unpredictably weaken, resulting in delays or dropouts.
The resulting algorithms can be applied to any type of unmanned vehicle—land, water, or air—or a mix of vehicles. Subbarao and his team will refine their work through simulations using small mobile robots that will run around their lab floor. Eventually, they will build aerial vehicles and validate their results in a confined test arena, such as the ones UTARI offers.
This research builds on a three-year grant Subbarao received from the Air Force Research Laboratory in 2016 to examine how time delays during information transfer can affect cooperating spacecraft. The new control algorithms he developed address how ground rovers can still successfully reach their destinations, talk to each other, and avoid collisions even if there is a short loss of network connectivity or the signal is delayed.
"The algorithm compensates for the delay, and the rovers still can perform the mission," Subbarao says. Now with the ONR grant, "we have to understand how those algorithms can be applied to aerial vehicles, and whether they can be applied with very little modification or if the extent of modification required has yet to be determined."
Improving Performance Through Networking
Emergency response is an application that would benefit from multiple vehicle coordination, allowing UAVs to perform more complicated missions. While a single UAV could be sent out to bring back images or video of, for example, a wildfire, vital time would be lost to downloading and processing the information. Multiple UAVs networked together, however, could send data in real-time, allowing emergency responders to respond faster and potentially saving lives and property.
Associate Professor Yan Wan
To date, most of the available hardware and software has been designed either for a single application or a single vehicle. But with the aid of a three-year, $998,803 NSF grant, Yan Wan, associate professor in the Department of Electrical Engineering, is building a generic system that researchers from various disciplines can use in their work involving networked UAVs. The flexible, modular design of this open-network aerial computing platform will provide an infrastructure for experts from fields like computer science, electrical engineering, and aerospace engineering to build on as they tackle components related to their own specific areas of interest. Such a system is expected to save both time and money.
Professor Frank Lewis
Dr. Wan's co-principal investigators on the project, which began in September 2017, hail from the University of Puerto Rico–Mayagüez, the University of North Texas, and Texas A&M University–Corpus Christi. Frank Lewis, electrical engineering professor and the Moncrief-O'Donnell Endowed Chair at UTARI, serves as an adviser. (Wan and Dr. Lewis are also collaborating on a four-year, $815,019 ONR project that began in June 2018 to develop performance guarantees for networked multi-body systems, using UAV pursuer-evader games as a case study.)
The first prototype is already available and includes a lightweight UAV-mounted computing device with connection ports and virtualization support, a UAV and antenna cooperative control system, and a broadband wireless communication system that allows multiple UAVs to network. The platform gathered tremendous attention when presented at the 2018 International Conference on Unmanned Aircraft Systems.
Increasing Safety for Those Below
While Amazon CEO Jeff Bezos made a splash with his 2013 announcement of a Prime Air service that would deliver packages to customers' doorsteps, there remain unanswered questions about how the widespread use of commercial drones will impact those on the ground.
To address these concerns, Wan and Subbarao are collaborating with Atilla Dogan, associate professor of aerospace engineering, on a $550,000 NSF project to quantify the risk UAVs pose as they fly over populated areas. The team will then develop an autonomous decision algorithm that will keep UAVs below a predetermined acceptable risk level as they execute their tasks.
Manfred Huber, professor in the Department of Computer Science and Engineering, and Brian Huff, associate professor in the Department of Industrial, Manufacturing, and Systems Engineering, also serve as co-principal investigators on the grant.
"Our objective is to increase public acceptance of unmanned aerial system (UAS) technology by demonstrating safe operation in various risk conditions, and to benefit commercial UAS use and the related job market," Dr. Dogan says.
Making UAVs Smarter
Meanwhile, at UTARI, researchers are working on new ways to improve UAV performance. At the June 2018 International Conference on Unmanned Aircraft Systems, Senior Research Scientist Aditya Das, Lundberg, and research scientist Hakki Sevil presented a paper on using groups of UAVs, called swarms, to locate an object of interest. One smarter UAV would fly at a higher altitude and perform a scan for the object, which could be anything from an infrastructure component, like an oil drill or fire hydrant, to an improvised explosive device. The vehicle would then deploy smaller, less intelligent drones to capture side and/or lower altitude views of the found object in higher definition, and from all that data create a 3D model, or orthophoto.
"It's meant to be a flexible system, where you could deploy as many vehicles as you need and build that 3D model onboard, so you could have it before the UAV landed to give to an inspector," Lundberg explains.
"Our relationships with industry and academic partners, as well as our FAA certification, make UTA and UTARI ideal for pursuing meaningful research in unmanned aerial systems."
Because UTARI has an FAA certificate of authorization to fly drones in the area around its facility, the researchers were able to conduct experiments outside, and they used drones to scan for an obscured trash can.
UTARI is also bringing the University's expertise in unmanned systems directly to business and industry. In a recent partnership, the Automation and Intelligent Systems division, led by Dr. Das, collaborated with simulation software company ZedaSoft Inc. UTARI helped the Fort Worth-based business develop its physical UAV capabilities by modifying a commercially bought drone, programming the autopilot, conducting systems checks, and educating company leaders about regulations, among other tasks. Lundberg also originally served as UAV pilot.
"Our relationships with industry and academic partners, as well as our FAA certification, make UTA and UTARI ideal for pursuing meaningful research in unmanned aerial systems," says Mickey McCabe, the executive director of UTARI.
Preparing Future UAV Innovators
Since 2014, UTA has been training its undergraduate and graduate students to serve as the next generation of experts in this burgeoning field through a multidisciplinary UAS certificate program. In this unique offering, students are team-taught in the introductory course by professors from a variety of engineering departments. In the second semester, the students split themselves into teams and build an unmanned vehicle to compete in a series of challenges.
In the unmanned systems industry, where the challenges are tightly woven together, the interdisciplinary experience students gain at UTA is invaluable.
Subbarao says that the programs increase their graduates' employment prospects. "The certificate puts the spotlight on them and their skill sets. When they go out into the unmanned systems industry, they'll have to work with people from other disciplines to improve the capability of the vehicles, and they already have that experience."
In the unmanned systems industry, where the challenges are tightly woven together, the interdisciplinary experience students gain at UTA is invaluable.
Will we one day get around via passenger drones? And will we actually have packages seamlessly dropped onto our doorsteps? Only time will tell if this Jetsons-like future will become a reality. But as the potential for unmanned aerial systems continues to take off, impacting everything from infrastructure to emergency response, UTA is ready to meet the accompanying challenges through its research and faculty know-how, industry and business partnerships, and focus on educating the next generation of experts.
