Small unmanned aerial vehicles (UAVs) have the potential to revolutionize various applications in civilian domain such as disaster management, search and rescue operations, law enforcement, precision agriculture, and package delivery. As the number of such UAVs rise, a robust and reliable traffic management is needed for their integration in national airspace system (NAS) to enable real-time, reliable, and safe operation. Management of UAVs traffic in NAS becomes quite challenging due to issues such as real-time path planning of large number of UAVs, communication delays, operational uncertainties, failures, and noncooperating agents. In this work, we present a novel UAV traffic management (UTM) architecture that enables the integration of such UAVs in NAS. A combined A*–mixed integer linear programming (MILP)-based solution is presented for initial path planning of multiple UAVs with individual mission requirements and dynamic constraints. We also present a distributed detect-and-avoid (DAA) algorithm based on the concept of resource allocation using a market-based approach. The results demonstrate the scalability, optimality, and ability of the proposed approach to provide feasible solutions that are versatile in dynamic environments.
Introduction
While unmanned air vehicles (UAVs) have been used in military operations for many years, they have recently generated a lot of interest due to their potential application in civilian domains. Applications include emergency management, law enforcement, precision agriculture, package delivery, infrastructure inspection, and imaging/surveillance [1–6]. As the use of UAVs rapidly becomes a reality in civilian domains, it becomes increasingly critical to solve the challenges emanating from integration of UAVs in the national airspace system (NAS). It is important for a UAV to plan its mission path, replan or adjust its trajectory to maintain separation with other aircraft. Traffic management adds a third goal: a UAV must act in a way that does not interfere with traffic. Furthermore, the dramatic increase in the number of aircraft, manned and unmanned, over the last 50 years will pose severe challenges to the current air traffic control. Hence, the radio technical commission for aviation and Federal Aviation Administration have been charged with a responsibility to implement a seamless change from air traffic control to air traffic management by 2020 [7–9]. To this end, the Federal Aviation Administration has begun to deploy the low altitude authorization and notification capability, which enables drone pilots access to controlled airspace near airports through a UAV service supplier [10]. This fits with the UAV traffic management (UTM) architecture developed by the National Aeronautics and Space Administration and industry.
National Aeronautics and Space Administration's TCL3 tests, held in spring of 2018, focused on testing technologies that maintain safe spacing between cooperative and noncooperative UAV over moderately populated areas. Their current UTM architecture includes an air navigation service provider, which interfaces with NAS data sources and provides constraints and directives to multiple industry UAV service suppliers, which coordinate with UAV operators, each other, and supplemental service providers to maintain a clear airspace [11].
For managing airspace traffic, there are various studies on lane-based systems [12], using rapidly exploring random trees and conflict bands [13], implicit coordinated detect-and-avoid using airspace tessellation, A*, and keep-out geofences [14]. Furthermore, there are market-inspired approaches for urban automotive intersection management [15], which could be expanded to work with lane-based or tessellation-based systems. While UTM may prevent some conflicts from occurring, there still needs to be detect-and-avoid (DAA) methods for UAVs. There are various methods for calculating escape trajectories that have been proposed for collision avoidance including classical control [16], fuzzy logic [17], E-field maneuver planning [18,19], game theory [20], evolutionary methods [21] and their application in numerical control machines path planning [22,23], automotive trajectory planning [24], and mixed-integer linear programming (MILP) [25,26].
The UTM problem essentially refers to planning paths of UAVs so that minimum separation is maintained between UAVs, static obstacles, and each UAV completes its mission. The UTM problem essentially involves two steps: (i) global path planning, which refers to obtaining collision-free paths for all UAVs in a global manner and (ii) DAA, which avoids collisions in a local manner. DAA becomes necessary because UAV traffic condition is subject to change due to any number of factors, including but not limited to: wind, new aircraft entering airspace, and changes in mission objectives. DAA's objective is to avoid collisions arising due to these dynamic effects and is usually implemented on a local basis. Criterion for assessing the performance of a DAA system includes how well safety was maintained, how close each UAV's trajectory is to optimal for its mission, and how quickly the system can resolve conflicts. In this paper, we utilize A*/MILP approach for global path planning. Use of A*/MILP ensures global optimality of the paths obtained. A* is first used to obtain initial trajectories of all the UAVs considering only static obstacles on a coarser grid scale. Paths obtained by A* are then refined by the MILP algorithm on a finer grid scale ensuring collision avoidance with other UAVs (dynamic obstacles). The DAA capability is implemented using a market-based approach. The DAA algorithm works by a market-based bidding process to resolve conflicts as they occur. This approach offers computationally efficient solutions, which are near-optimal, and can be implemented in a distributed fashion. Safety is quantified by the difference between the closest point between UAVs at a given time and any loss of separation. The problem is formulated such that several UAVs fly through a three-dimensional representation of a city, following a set of waypoints. The approaches proposed in this paper have been validated using very realistic numerical simulations that include three-dimensional (3D) model of downtown part of City of Cincinnati, OH.
The paper is outlined as follows: We propose a novel UTM architecture that is composed of flight protocol, communication aspects, and the navigation scenario in Sec. 2. Section 3 talks about the global path planning problem formulation and its solution and Sec. 4 explains our market-based DAA approach. We present the simulation results in Sec. 5 and the discussions in Sec. 6.
Proposed Unmanned Aerial Vehicles Traffic Management Architecture
We present a novel UTM architecture in this section that comprises the flight, communication, and navigation protocol. The nomenclature of various terms is defined as follows:
UAS: Unmanned aerial system, also referred to as UAV, is an autonomous flying vehicle. It is capable of talking to the UAS operator via wireless telemetry, using LTE, Wi-Fi, or any other methods. It is also equipped with an autopilot and an automated dependence surveillance-broadcast.
UAS operator: A person, or an organization, that is in charge of flying vehicles and is responsible for ensuring airworthiness by proper maintenance.
NAS: National airspace system is the airspace, its facilities, and airports of the United States along with their rules, regulations, policies, personnel, etc.
VLOS: Visual line of sight, basically means that the system can be observed visually. The contrary term is BVLOS (Beyond VLOS), which means the system is not visible to an observer.
UTM: UAS traffic management service is the main service that monitors and ensures that all the airborne UAS are operating in a safe manner. It also plans the flight path for these UAS as requested by the user through a ground control station. The UTM talks to ground control station through the internet to receive mission plan requests and responds with an approval or denial. The UTM also talks to the airspace management database that is shared with the air traffic controllers for manned aircrafts.
No-fly zone: A geo-fenced area that is restricted for general UAS to enter. This could be permanent, such as 400 ft. above ground level or airports, or it could be temporary, such as construction or accident sites.
Waypoints: Geo-coordinates of a location that will be visited by a UAS during its flight. They could be mandatory or optional.
Flight mission: A layout of the flight plan with starting and ending geolocations, optionally but generally, with intermediate waypoints. It is basically trajectory information with timestamped waypoints to be visited. Additionally, it might include the tolerance around the area and/or the time where/when the UAS could be located.
Centralized approach: This kind of approach means that all the operations are handled by a central service.
Decentralized approach: Opposed to centralized approach, the operations are handled by the services that are responsible for their respective coverage areas. Essentially, this is a combination of several centralized approaches talking to each other.
UAS state: The state of the UAS, which includes the position and velocity information and additionally, may include the health of the UAS, such as battery status, sensor health, etc.
Obstacles: The locations in the environment that cannot be taken by the UAS. These could be buildings, trees, and any other flying objects, including other UAS.
Noncooperative/rogue UAS: The UAS, which have deviated from the prescribed flight plan by the UTM and behaving in a suspicious manner. This could be caused by system malfunction or malicious intent of the user.
Pop-up threats: Threat to the UAS navigation that was not anticipated. This could include rogue UAV, or even unplanned construction/accident sites.
C2: Command and control, the method to send control commands to the UAS to ensure it follows the prescribed flight plan.
SAA: Sense and avoid is the capability to sense the obstacles (static or dynamic) and use obstacle avoidance algorithms to avoid collisions with those obstacles by flying around them at a safe operating distance.
Flight Protocol.
A schematic of the flight protocol is shown in Fig. 1. Sequentially, this can be realized as follows:
- (1)
The UAS operator comes up with a desired mission. This demand would include a starting point, a destination (which could be same as the starting point), optionally, the waypoints to visit, the time of flight, and additionally, some identification of their UAS, such as build type, type of sensors, and capabilities.
- (2)
The UAS operator sends this demand to the UTM service.
- (3)
The UTM responds with an approval or denial. If approved, a detailed flight mission plan is returned. If denied, the user may retry with a new request.
- (4)
If the UAS operator received an approved flight plan, they can communicate this to their UAS through wireless link.
- (5)
The UAS starts the mission according to the given flight plan.
- (6)
If the actual navigation performance of the UAS deviates from the required navigation performance by a prescribed limit, the UAS is considered as noncooperative and goes into a rogue state. An occasional small deviation could be a result of noise in sensors or positioning and may be ignored.
- (7)
In case of rogue UAS, the UTM sends the mission abort commands to the UAS directly, thus forcing the UAS to end mission and land at the nearest safe location.
- (8)
In case the UAS operator comes up with a need to modify the original flight plan while the UAS is still flying, they may request this to the UTM and an updated flight mission may be received. This change can be uploaded to the UAS directly or sent through the UAS operator.
- (9)
If the UAS completes the mission successfully, the UAS operator sends a mission report log and the UTM retains it.
- (10)
Once the mission report is received, the UTM stops monitoring that UAS and the UAS operator is unsubscribed from the UTM service until next mission.
Navigation Protocol.
A schematic of the navigation protocol is shown in Fig. 2. The airspace below 400 ft. above ground level will be divided into N number of layers of an appropriate height depending on minimum safe operating radius. As the UAS operators request a flight path, a cuboid strip of airspace from the starting location to destination will be allocated to the user for the requested time. These airstrips will be divided into separate sections that will be reserved for the operator for a given time. This would mean that at any time instant, the UAS can be located within a given location only (with tolerances on location and time). This would make the airspace utilization more efficient by freeing up sections through which the UAS has already passed.
Communication Protocol.
A schematic of the communication protocol is shown in Fig. 3. We define the various communication links as follows:
The primary command and control (C2) link will be set up between the UTM server and the UTM client onboard the UAS. This will be the primary mode of communication between the UTM service and the UAS. The UTM will get the state of the UAS through this link that will include position, velocity, and the health of the UAS (such as battery status, and sensor status).
There will be a secondary C2 link between the UAS operator and the UAS, which may be used if needed. For instance, the onboard autopilot will be aware of the environment and strategies for a successful mission through the primary C2. However, there might be cases when sudden changes in environment arise, such as accident sites and medevac instances, which would require the UAS to change its mission course. The UAS will then ask the operator to supply with proper strategies to continue its mission. Another example could be if the operator decides to abort the mission prematurely, they can send the abort commands directly to the UAS.
The UTM to UAS Operator communication will be the one instantiating/subscribing to the UTM service. This would include mostly environmental updates, such as weather info, terrain updates, and emergency announcements.
The UAS will be able to subscribe to other agents' positions through an onboard position information broadcast system. This will include the position and velocity information of the agents that will be vital in making SAA decisions.
Global Path Planning—Problem Formulation and Solution Methodology
This paper focuses on the development of a method to manage UAV traffic in an urban environment. The UAVs must navigate through an environment full of static and dynamic obstacles (in the form of each other). The path-planning problem is solved using a combination of A*, to find a general optimal path, and MILP to find optimized trajectories with respect to the static and dynamic obstacles. Particularly, A* provides the shortest path for all the UAVs at macro level (at a coarser grid scale) while the MILP part of the algorithm gives the optimal path in terms of minimum energy consumption of all the UAVs at a finer grid scale. In what follows, we describe our A*/MILP approach in more detail.
Obstacles drawn from laser imaging, detection and ranging data of a city's height are represented by 8 m × 8 n rectangles, where m and n are nonzero integers. Each obstacle has minimum and maximum x and y coordinates. Each vehicle has a minimum and maximum speed, a mass, and a maximum acceleration. Each vehicle submits a flight plan at a specified time with a start point, end point, and K destinations, where K is a non-negative integer. There are two major parts to the algorithm: an A* portion that conducts macro path-planning considering only obstacles, and a MILP portion that finds an optimal path considering the waypoints from A*, time, vehicle dynamics, obstacle positions, and the other vehicles in the airspace. The overall algorithm for the UTM problem is shown in Algorithm 1.
foreach flight plan (from earliest takeoff to latest)do |
foreach pair of waypoints in the flight plando |
Path = A*(pair of waypoints) |
end |
Construct the waypoints in path into segments short enough for the s-UAS to traverse in the time given for a MILP call. |
Trajectory = empty set of x, y, t points |
for each segment do |
Find the segment trajectory using MILP given the waypoints in the segment and the intruders in the segment for the time provided |
Append the segment trajectory to the current trajectory |
end |
Add the new trajectory to the list of trajectories |
end |
foreach flight plan (from earliest takeoff to latest)do |
foreach pair of waypoints in the flight plando |
Path = A*(pair of waypoints) |
end |
Construct the waypoints in path into segments short enough for the s-UAS to traverse in the time given for a MILP call. |
Trajectory = empty set of x, y, t points |
for each segment do |
Find the segment trajectory using MILP given the waypoints in the segment and the intruders in the segment for the time provided |
Append the segment trajectory to the current trajectory |
end |
Add the new trajectory to the list of trajectories |
end |
Initialize the start node with starting coordinates, no parent, and g and f scores of 0 |
Initialize all other Nodes with no parent and g and f scores of infinity |
Declare the open set as an empty priority Queue |
Declare the closed set as an empty set |
Add the start node to the open set |
Add all invalid nodes from the grid to the closed set |
whilethe open set is not emptydo |
Pop the best node from the open set |
Add the best node to the closed set |
ifthe best node is the goalthen |
Declare the Path as an empty set |
Current node = Best node |
whileCurrent node has a parentdo |
Prepend the current node to the Path |
Current node = parent(Current node) |
end |
Return Path |
end |
foreach neighbor of the best nodedo |
ifneighbor is not in closed setthen |
tentativeGScore = best node's g score + distance(Best node, neighbor) |
iftentativeGScore < neighbor's g Score |
then |
neighbor's g score = tentativeGScore |
neighbor's f score = neighbor's g score + distance(neighbor, goal) |
parent(neighbor) = best node |
Update the neighbor in the open set |
end |
end |
end |
end |
Return path unavailable message |
Initialize the start node with starting coordinates, no parent, and g and f scores of 0 |
Initialize all other Nodes with no parent and g and f scores of infinity |
Declare the open set as an empty priority Queue |
Declare the closed set as an empty set |
Add the start node to the open set |
Add all invalid nodes from the grid to the closed set |
whilethe open set is not emptydo |
Pop the best node from the open set |
Add the best node to the closed set |
ifthe best node is the goalthen |
Declare the Path as an empty set |
Current node = Best node |
whileCurrent node has a parentdo |
Prepend the current node to the Path |
Current node = parent(Current node) |
end |
Return Path |
end |
foreach neighbor of the best nodedo |
ifneighbor is not in closed setthen |
tentativeGScore = best node's g score + distance(Best node, neighbor) |
iftentativeGScore < neighbor's g Score |
then |
neighbor's g score = tentativeGScore |
neighbor's f score = neighbor's g score + distance(neighbor, goal) |
parent(neighbor) = best node |
Update the neighbor in the open set |
end |
end |
end |
end |
Return path unavailable message |
It may be noted that we include acceleration in our cost function in order to minimize force required for the motion, which translates into energy consumption.
Detect-and-Avoid—Market-Based Approach
As discussed in Sec. 3, the path planning of the UAVs is done based on a globally optimized solution by the UTM when they request a mission. This means there will not be any conflicts or chances of UAVs breaching safe zones of other UAVs. However, due to unavoidable circumstances and practical considerations, sometimes a UAV may deviate from its path. This may lead to conflicts with other UAVs.
The DAA process is triggered as soon as the UAVs enter a conflict zone. This conflict zone is determined based on safe operating distance of UAVs, which, in turn, may be based on the individual capabilities of the UAVs, such as maximum velocity and agility. Thus, the conflict zone is tessellated into grids and each grid cell may be envisioned as a resource that an agent (UAV) is trying to access at a given time. We borrow the concepts from economic markets such as demand supply for price updates and agent-level calculations [28]. This ensures scalability, optimality, and ability to provide feasible solutions that are fast enough to be responsive for dynamic changes.
Implementation.
In the above equations, i and j are the resources (cells) and the agents (UAVs), respectively. Hence, Eq. (13) represents the total profit accumulated by all the UAVs. na is the number of UAVs and nr is the number of resources (grid cells). aij,t represents the known profit of agent j in utilizing resource i at time t. xij,t is the decision binary variable, which is 1 when the resource i is being utilized by the agent j, and 0, otherwise. ts and tr represent the start time of the conflict and the time instant when all conflicts have been resolved, respectively. In this paper, we consider a constant value of ai,j,t, which essentially means that the cost function is a measure of total distance traveled by all the UAVs. Equation (14) ensures that all UAVs utilize a resource at a given time (i.e., continuity) and Eq. (15) prevents a resource from being utilized by more than Ni agents. For collision avoidance, Ni would be equal to unity.
Simulation Results
Simulation Environment.
We consider an environment of 1.25 mi × 1.25 mi with a maximum height of 500 ft (2 km × 2 km × 152 m) in the downtown area of the city of Cincinnati. The laser imaging, detection and ranging data available from the USGA [29] were used to generate 3D model of the downtown area of Cincinnati. The UAVs are allowed to fly between 50 ft and 500 ft only. The grid size is chosen as 8 m × 8 m × 8 m. Multiple UAVs with random start and destination locations start their mission at different times. The initial path planning has been described in Sec. 3. Figure 5 shows the simulation area with height map of obstacles. To introduce operational delays in the system, we have assumed that 10% of the UAVs get delayed randomly by a uniformly distributed delay period of a maximum of ten time steps. This essentially means that some of the UAVs will start later than their proposed time of start. The market-based DAA algorithm can be constructed as in Algorithm 3.
Generate grids and set all grid prices to zero |
Set tglobal = 0 |
whilemissions remaindo |
Set tglobal = tglobal + 1 |
Detect conflicts in next tdetect time steps |
whileconflicts remaindo |
Update grid prices (Eq. (17)) |
Replan UAV paths between current position and local goal* based on Eq. (16) |
end |
end |
*local goal is the grid cell where UAV would have been after tdetect time steps |
Generate grids and set all grid prices to zero |
Set tglobal = 0 |
whilemissions remaindo |
Set tglobal = tglobal + 1 |
Detect conflicts in next tdetect time steps |
whileconflicts remaindo |
Update grid prices (Eq. (17)) |
Replan UAV paths between current position and local goal* based on Eq. (16) |
end |
end |
*local goal is the grid cell where UAV would have been after tdetect time steps |
Results.
We simulated a number of scenarios for a different number of UAVs with different start and destination positions. Here, we present three such scenarios for 10, 25, and 100 UAVs. For 10 UAVs, we have considered a smaller area, to ensure there are conflicts observed and to demonstrate the potential of our DAA algorithm. This is shown in Fig. 6. The different UAV paths are shown in different colors. The destination of each UAV is shown by a ×. It should be noted that the axes represent the grids in the figures. The conflicting cell is identified by the Δ. To demonstrate the applicability of the proposed method to larger number of UAVs, Figs. 7 and 8 show the path planning over the whole simulation environment for 25 and 100 UAVs, respectively.
Optimality Validation.
In order to demonstrate the optimality of proposed market-based DAA approach, we have compared the results obtained from market-based approach with those obtained from MILP solution, proposed in Refs. [25] and [30], as shown in Table 1. For a given tdetect in the conflict zone, we formulated the conflict zone and the UAVs in MILP framework and found that the result provides compatibility with the MILP solutions. Table 1 represents the comparison of three incidents of the scenarios provided in Sec. 5.2 and the MILP solver.
Distance traveled by all the UAVs (number of grid cells) | Market-based approach | MILP approach |
---|---|---|
Incident 1 | 26 | 26 |
Incident 2 | 28 | 28 |
Incident 3 | 28 | 28 |
Distance traveled by all the UAVs (number of grid cells) | Market-based approach | MILP approach |
---|---|---|
Incident 1 | 26 | 26 |
Incident 2 | 28 | 28 |
Incident 3 | 28 | 28 |
In Table 1, for each of the incidents, we have reported the total distance traveled by all the UAVs, engaged in the conflict. The results represent compatibility with the MILP solver, which verifies the optimality of the proposed DAA solution.
Discussions
In this work, we have developed a centralized UTM architecture for safe path planning and navigation of multiple UAVs in national airspace. We have identified essential flight, navigation, and communication protocols for managing multiple UAV mission planning by the UTM server.
Further, we present a MILP-based solution for the initial path planning of the UAVs and a distributed DAA algorithm inspired from the concept of resource allocation to various tasks in an optimized way from market-based approach. We demonstrated that this concept can be extended to enable a safe navigation for multiple UAVs in a given environment, while generating a situational awareness in the region using on-board and ground-based sensors.
We also simulated a general scenario of multiple UAVs requesting mission plans with different start and destination locations. The simulation environment is taken as a subset of the downtown area of the city of Cincinnati, OH. To simulate the operational delays, we have introduced delays in the starting time of some of the UAVs. The simulation results corroborate the validity and scalability of our approach, and it can be observed that this approach can be easily extended to a large number of UAVs in a given environment while optimally carrying out the path planning and DAA in a safe and reliable manner. The developed algorithm is also robust to delays, uncertainties, and deviations in the UAV paths. A striking feature of this DAA approach is the ability to globally optimize the cost function of all the UAVs present in the environment as validated in Sec. 5.2.1. This makes it suitable for real-time path planning and DAA in uncertain environments where UAVs enter and leave at will.