Control

A Decentralized Cooperative Control Framework for Multiple UAVs
by Dustin Michael Geletko Thesis submitted to the College of Engineering and Mineral Resources at West Virginia University in partial fulllment of the requirements for the degree of Master of Science in Electrical Engineering Asad Davari, Ph.D. Tim Menzies, Ph.D. Bojan Cukic, Ph.D., Chair Lane Department of Computer Science and Electrical Engineering Morgantown, West Virginia 2007 Keywords: Multiple UAVs, Swarm, Decentralized, Cooperative Control, Reactive Behavior, Task Allocation, Path Planning
Copyright 2007 Dustin Michael Geletko
Abstract
A Decentralized Cooperative Control Framework for Multiple UAVs by Dustin Michael Geletko
The main focus of this research and thesis was the development of a decentralized framework for the cooperative control of multiple unmanned aerial vehicles (UAVs). The goal of the software was to coordinate multiple UAVs to perform a wide area search on a dened area. The developed algorithms were based on the idea that simple rules can be programmed leading to an overall complex behavior. The simple algorithm that was developed illustrates a high level of autonomy, including reactive behavior without human interaction. The developed software was tested and run onboard an actual hardware avionics unit with a payload computer in the loop.
iii
Acknowledgments
This research and development was conducted while employed with Alion Science and Technology under Naval Air Systems Command (NAVAIR) contract #N00421-05-D0016. The reference herein to any specic commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not constitute or imply its endorsement by the United States Government or Alion Science and Technology. I would like to thank Alion Science and Technology, Augusta Systems, WVU Tech, and NAVAIR for the opportunity to be involved with this exciting research project and providing both professional and technical advice throughout my employment. I would also like to thank all of my committee members for their guidance, assistance, and patience during my graduate studies. Dr. Cukic, Dr. Menzies, and Dr. Davari have always provided invaluable technical advice and it was a privilege to work under such outstanding researchers. Finally, my deepest gratitude goes to my family for their love and support with all of my endeavors and decisions. A special thanks goes to my wife Jenn for her patience, encouragement, and support during the late nights of hard work. I would like to sincerely thank my parents for supporting me throughout my life and showing the value of hard work and that any goal could be accomplished.
iv
Contents
Title Page Abstract Acknowledgments Contents List of Figures List of Algorithms Notation and Acronyms 1 Introduction 2 Review of Literature 2.1 Trajectory Generation . . . . . . . . . . . . . . 2.2 Path Planning . . . . . . . . . . . . . . . . . . 2.2.1 Serpentine Patterns . . . . . . . . . . . 2.2.2 Graph Based Methods . . . . . . . . . . 2.2.3 Stigmergic Approaches . . . . . . . . . . 2.2.3.1 Pheromone-Based Approaches 2.2.3.2 Physics-Based Approaches . . 2.2.3.3 Other Approaches . . . . . . . 2.3 Task Allocation . . . . . . . . . . . . . . . . . . 3 Investigation 3.1 Algorithm Overview . . . . . . . . . . . 3.1.1 Operational Requirements . . . . 3.1.1.1 Previous Requirements 3.1.1.2 Current Requirements . 3.1.1.3 Summary . . . . . . . . 3.1.2 System Requirements . . . . . . 3.1.3 Architecture . . . . . . . . . . . 3.1.4 Overall System Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i ii iii iv vi vii viii 1 7 8 9 9 10 11 11 12 13 13 15 15 15 15 17 20 20 21 25
CONTENTS 3.2 3.3 3.4 3.5 Software Initialization . . . . . . . . Network . . . . . . . . . . . . . . . . Avionics Interface . . . . . . . . . . Environment . . . . . . . . . . . . . 3.5.1 Search Area Information . . . 3.5.2 Target Information . . . . . . 3.5.3 Swarm Information . . . . . . Path Planning . . . . . . . . . . . . 3.6.1 Search Path Planning . . . . 3.6.2 Identication Path Planning Task Allocation . . . . . . . . . . . . 3.7.1 Description . . . . . . . . . . 3.7.2 Cost Function . . . . . . . . . ATR Simulation Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
v 28 31 37 39 40 43 44 44 50 52 54 56 58 58 62 62 62 64 67 67 69
3.6
3.7
3.8
4 Summary and Conclusions 4.1 Algorithm Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 Search Path Planning Tests . . . . . . . . . . . . . . . . . . . . . . . 4.1.2 Complete System Tests . . . . . . . . . . . . . . . . . . . . . . . . . 5 Recommendations 5.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References
vi
List of Figures
1.1 1.2 1.3 2.1 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22 4.1 4.2 4.3 5.1 UAS Autonomy Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Swarm Environmental Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . Path Planning Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . Outer Loop Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sweeping Search Pattern Setup . . . . . . . Demonstration Search Area . . . . . . . . . System Package Diagram . . . . . . . . . . Algorithm Class Diagram . . . . . . . . . . Overall System Operation . . . . . . . . . . Overall System State Diagram . . . . . . . Sample XML Software Initialization File . . Software Initialization State Diagram . . . . Network Communications Class Diagram . Network Messaging State Diagram . . . . . Avionics Messaging State Diagram . . . . . Environment Class Diagram . . . . . . . . . Search Area Storage . . . . . . . . . . . . . Search Area Calculations . . . . . . . . . . Planning State Diagram . . . . . . . . . . . Search Area Path Planning . . . . . . . . . Search Path Planning State Diagram . . . . Identication Path Planning . . . . . . . . . Identication Path Planning State Diagram Task Allocation Illustration . . . . . . . . . Task Allocation State Diagram . . . . . . . ATR Simulation Software Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4 5 9 16 19 22 24 25 27 28 30 34 36 38 40 41 43 47 51 53 54 55 57 59 61 64 65 66 68
Percentage of Search Area Covered . . . . . . . . . . . . . . . . . . . . . . . Search Area Grid Count with Priority Areas . . . . . . . . . . . . . . . . . . Full System Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Improved Identication Path Plan . . . . . . . . . . . . . . . . . . . . . . .
vii
List of Algorithms
1 2 3 4 5 Obtaining Correct Grid Structure . . . . . . . . . . . . . . . . . . . . . . . . Cooperative Control Algorithm Main Loop . . . . . . . . . . . . . . . . . . Avionics Interface Thread . . . . . . . . . . . . . . . . . . . . . . . . . . . . Swarm Server Thread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Task Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 48 49 49 56
viii
Notation and Acronyms

AC AOI AP API ATR COTS DAI DLL FMS FOV FP GPS GCS GUI HIL ID ISR NFZ OI OS SDK TA TCP/IP UAV WP Aircraft Area of Interest Autopilot Application Program Interface Automatic Target Recognition Commercial O-The-Shelf Distributed Articial Intelligenct Dynamic Link Library Flight Management System Field of View Flight Plan Global Positioning System Ground Control Station Graphical User Interface Hardware-in-the-Loop Identication Intelligence, Surveillance, and Reconnaissance No Fly Zone Operator Interface Operating System Software Development Kit Task Allocation Transmission Control Protocol Unmanned Aerial Vehicle Waypoint
Chapter 1
Introduction
Unmanned aerial vehicles (UAVs) have been used for reconnaissance missions in past years in situations such as Bosnia, the Persian Gulf, and Afghanistan. UAVs have been a valuable tool in the ght against terrorism, and the demand for these benecial assets will only increase. Motivation to develop UAVs was driven by the advantages they provide over piloted vehicles, which include: 1. Maximum Maneuverability 2. Low Human Operator Risk 3. Weight Savings 4. Lower Overall Cost 5. Superior Coordination 6. Potential to Complete New Operational Missions as stated in [1]. Examples of some missions that could be benecially assigned to UAVs include surveillance/reconnaissance, communications relays, and radar jamming devices. The topic of this research addressess the surveillance/reconnaissance role of UAVs. The Department of Defense established the UAV Joint Project Oce (JPO), which shifted the focus from large-scale UAVs to the development of small, low altitude, and low cost UAVs that were globally accepted as disposable [2]. The implementation of multiple UAVs on a battleeld to complete certain missions has tremendous potential but is only feasible for these low cost, dispensable aircraft. It would prove advantageous to develop a
2 control mechanism for these UAVs that would allow them to be launched in large numbers, each having individual agendas, and cooperating to accomplish a large scale mission. This goal is loosely dened as a UAV swarm. A formal denition of swarming with respect to UAVs, provided in [3] is a collection of autonomous individuals relying on local sensing and reactive behaviors interacting such that a global behavior emerges from the interactions. Although the term low cost is subjective, current aircraft are far too costly to deploy in large numbers. The operating manpower for an individual UAV is also tremendous. As summarized in [4], several operators are typically required to control todays UAVs, but future UAVs should be designed to allow a single operator to control an entire eet. In future systems, the goal is to have a swarm of expendable UAVs completing a mission with little interaction from human operators. Operators could concentrate solely on the information provided by the swarm at a macro level instead of the control of each individual UAV at the micro level. When the aircrafts are truely low cost, the failure/loss of an individual UAV would become irrelevant and not aect the overall mission. Most modern UAVs employ extremely complex control strategies. Unfortunately, the requirement of a small, disposable, and inexpensive UAV hinders the use of powerful onboard computers. Simple control mechanisms must currently be deployed that operate within the limitations of the onboard payload computers, the typically low-resolution sensors, and limited communications bandwidth. A lot of research has beed conducted on various control strategies, which are elaborated in section 2. Unfortunately, most of the current research is lab oriented and not practical for operational usage onboard modern small-scale UAVs due to processing and communications limitations. As stated in [5], An examination of the state-of-the art in swarm intelligence applications in the academic, commercial, and military research community indicates that this approach to robotic control may be achieving the critical mass necessary to exit the laboratory, but, despite its promise, swarm robotic control software is technically far behind its target hardware. The coordinating and controlling tasks must be performed onboard the aircraft payload computer in real-time, which makes the process dicult because of other limiting factors and problems. The major challenges aecting the cooperation and control of swarms of UAVs include bandwidth limited in-ight communications systems, a coupling eect of vehicles, and the time taken to achieve required levels of coordination [1]. Clearly, every vehicle cannot send all sensor information regarding the environment due to the limiting factors of the communication system, and a change of a single UAV may aect other UAVs
3 in the proximity due to the strong coupling of the aircraft. This shortcoming leads to the purpose of this research. The goal of this project was to develop a decentralized cooperative control algorithm for multiple UAVs. Each of these key requirements and denitions will be discussed in subsequent sections. It should be stressed that although this algorithm may be extremely simplied and not nearly as complex as current research in the eld, the algorithm provides an operationally eective solution that could be immediately utilized onboard existing UAVs. In fact, actual UAV systems were used, including avionics, sensors, and payload computers, to test the algorithm. It is envisioned that this research would provide a foundation in which more complex behaviors could be built. The algorithm, even in its simplicity, illustrates an advanced level of autonomy. The various levels of autonomy, as dened in [6], are illustrated in Figure 1.1. Although the specic denitions of each autonomous control level are not clearly dened, it can be seen that this research meets many of the advanced autonomy requirements, such as distributed
UAS ROADMAP 2005
control, group coordination, and onboard route replanning.

flight. Figure 4.0-2 depicts where example UA stand in comparison to their ten levels of autonomy.
Autonomous Control Levels
Fully Autonomous Swarms Group Strategic Goals Distributed Control Group Tactical Goals Group Tactical Replan Group Coordination Onboard Route Replan Adapt to Failures & Flight Conditions Real Time Health/Diagnosis Remotely Guided 10 9
UCAR Goal
8 7 6 5 4 3 2
Pioneer Global Hawk, Shadow, ER/MP, and Fire Scout Predator J-UCAS Goal
1955
1965
1975
1985
1995
2005
2015
2025
FIGURE 4.0-2. TREND IN UA AUTONOMY.
4.1
Figure 1.1: UAS Autonomy Levels PROCESSOR TECHNOLOGIES
Although today's processors allow UA to fly entire missions with little or no human intervention, if the ultimate goal is to replace a pilot with a mechanical facsimile of equal or superior thinking speed, There are a responses (algorithms) concepts that and experience, then consideration when memory capacity, andcouple of general gained from trainingwere taken into processors of human-like speed, memory, and situational adaptability are necessary. Human capabilities are generally designing theequate to 100 million million-instructions-per-secondFirst, the goal was to develop a totally control framework for multiple UAVs. (MIPS) in speed and 100 million agreed to megabytes (MB) in memory. In the 1980s, AFRL attempted to develop a robotic adjunct to a fighter pilot under the Pilot's Associate program, but the available processor technology proved insufficient. Figures 4.1-1 and 4.1-2 illustrate the progress in processor technology toward human levels of performance that has occurred and that are likely to be seen in the coming 25 years. Both show that today's supercomputers' are within a factor of 10 of achieving human equivalence in speed and capacity and could achieve human parity by the 2015 timeframe. The cost of a supercomputer is however uncompetitive with that of a trained human, but by 2030 the cost of a 100 million MIP processor should approach $10,000. As for inculcating a fighter pilot's training and experience into a robot brain, the equivalent of Top Gun school for tomorrow's J-UCAS will consist of a post-flight download in seconds. Today's silicon-based semiconductor processors will be limited to features about 0.1 micron in size, the so-called "point one limit," by current manufacturing techniques based on ultraviolet lithography. Once
4 decentralized algorithm, which was a major requirement. A majority of the current work in this eld rely on powerful central computers to perform processing. A centralized computer frequently leads to exponential increases in communication bandwidth requirements and the size of the controlling software [7]. Other disadvantages of a centralized control algorithm are that the central system makes the algorithm vulnerable to failure, it can easily become a performance bottleneck, the operation cannot be expanded beyond the intentional boundary, and it is usually a large complex software artifact that is dicult to understand and maintain [8]. The proposed algorithm relies on local information about the environment and inter-aircraft communications to accomplish mission goals, thus eliminating system failure upon single point failure. The other main design goal was to provide cooperative control among the UAVs. Because an environment can quickly change, the algorithm should possess reactive behavior, which is dened as the ability of agents to perceive their environment and respond to changes [9]. Reactive behavior provides a more robust system. After all, every tactical situation and scenario cannot be predicted and accounted for while programming desired functionality. While interacting with the environment, certain events should stimulate the UAVs to perform certain tasks, possibly cooperating in decision making. Figure 1.2, taken from [9], illustrates the non-terminating interaction between the entity and the environment.
Figure 1.2: Swarm Environmental Inputs The term control is used loosely in the context of this research. The UAVs are indeed controlled, but not in the context of traditional control theory. The entire scope of this research involves the path planning between the UAVs. A traditional path planning architecture is illustrated in Figure 1.3, taken from [10]. This research solely focuses on the
Submitted to the 2003 IEEE Conference on Decision and Control
ultiple UAV Cooperative Search under Collision nce and Limited Range Communication Constraints
Cooperative Waypoint Path Planner. It is assumed that once the waypoints are dened, the low-level autopilot will follow the generated path using traditional control laws. In order
to task allocation was also Randal W. Beard1 illustrate reactive properties,Timothy W. McLain utilized in the algorithm to assign rical and Computer Engineering jobs. AlthoughMechanical Engineering UAVs various both topics of path planning and task allocation could be Brigham Young University Brigham Young University researched in great detail separately, simple implementations were developed due to the Provo, Utah 84602 Provo, Utah 84602 tmclain@et.byu.edu beard@ee.byu.edu scope of the project.
Abstract
focuses on the problem of cooperatively ing a team of unmanned air vehicles ea of interest that contains regions of opregions of potential hazard. The objecAV team is to visit as many opportunities hile avoiding as many hazards as possible. operation, the UAVs are constrained to mmunication range of one another. Colce is also required. Algorithms for teamndividually-optimal/team-suboptimal soeveloped and their computational comared. Simulation results demonstrating of the cooperative search algorithms are 1 Introduction
Cooperative Waypoint Path Planner J

Dynamic Trajectory Smoother J
Trajectory Tracker J
Low-Level Autopilot K Unmanned Air Vehicle (UAV)
e problem where a team of unmanned air Vs) is given the task of searching a region Figure 1: Path Planning Architecture. n opportunities and hazards. We assume Figure 1.3: Path Planning Architecture V is equipped with sensing capability that ons of opportunity and regions of hazard path planner (CWPP) produces waypoint paths for iate look-ahead window of the UAV. We the UAVs operating environments for constraints. Dierent conceptual which satisfy the cooperation the proposed swarm technology hat the team of UAVs is equipped with The dynamic trajectory smoother (DTS) smoothes tion network, and that the connectivity were researched. The goal the waypoint paths producing time parameterthrough of this project was to develop a framework for decentralized k depends upon the relative distance beized trajectories on performing a cooperation conoring UAVs. Therefore, maintaining netcontrol of multiple UAVs, focusing which maintain the wide area target search in a dened straints and also satisfy the kinematic constraints of ivity constrains the maximum allowable area of interest the The focus was to develop an (TT) uses inereen UAVs. In addition, we assume that (AOI). UAV [3, 4]. The trajectory tracker autonomous decentralized onboard tial position and heading information to track the tras essentially two dimensional so collision cooperative behavior algorithm thatby the DTS, and outputs desired alti- tasks to the aircraft jectory produced dynamically sent ight plans and st be accounted for explicitly. The contude, velocity, environment. The environment consisted of events from based on various inputs from theand heading commands [5]. The physical for the team is to maximize the regions of UAV is controlled by a low-level autopilot which mainisited by the team, while minimizing the tains commanded altitude, velocity and heading [6]. ard visited by the team, the avionics, onboard sensor, and messages from other nodes of the swarm, as illustrated in subject to two In our approach, cooperation constraints such as colnts: (1) that the communication network Figure 1.2. Possible lision avoidance andaircraft, ground stations, and simulation computers. nodes were other maintaining communication conected at all times, and (2) that there are nectivity, are handled by certain requirements while The goal was to provide an algorithm that metthe cooperative waypoint performing a wide between UAVs. path planner. The primary contribution of this paper vious work on cooperative control of area random search is to describe several designs for would be further divided into regions of of a specied area. The area the cooperative path we have used the path planning archiplanner given collision avoidance and limited commuin Figure 1. The cooperative waypoint various priorities. Regions of higher priority would have to be searched a specied number nication constraints. Recent work in of low priority. This single UAVs ng author. of times more frequently than areas trajectory generation forbasic implementation could then
6 be expanded to provide more advanced capabilities in future versions. Continuous testing could be performed, and individual modules could be revised until satisfactory results were obtained.
Chapter 2
Review of Literature
Much research has been conducted in the area of multiple UAV control, mainly because of the realized potential to benet the military. The described work is limited to applications and missions related to the project, namely Intelligence, Surveillance, and Reconnaissance (ISR) missions. According to [11], although agents can search an environment without cooperation, a more ecient search involves cooperation to minimize duplicate coverage. The scope of this project and research can be divided into the following two subtopics: 1. Path Planning 2. Task Allocation Path planning involves calculating and sending a ight plan to the autopilot. In the context of this research, task allocation involves calculating the optimal identication aircraft to service, or identify, a particular target. The eld of UAV swarming is relatively new and is only recently evolving. Previously, most research involved pre-mission planning. The major disadvantages of a centralized pre-mission planner are obvious. This method could provide a globally optimum solution assuming the environment is completely deterministic. Unfortunately, the battleeld is an uncertain and rapidly changing environment [12], which reinforces the need for a dynamic and reactive path planning system. Intelligent algorithms should be developed to run onboard the UAV. The current trend in cooperative control is to distribute the decision making among the vehicles, which leads to a more robust algorithm with the ability for dynamic role assignment [13].
2.1
Trajectory Generation
There has been much research on trajectory generation. Although trajectory gen-
eration is beyond the scope of this project, it is briey mentioned because future improvements on this software could involve generating more complex paths to possible evade targets. A majority of the path planning research also involves a start and goal location. This again illustrates the focus on pre-mission planning, which is not sucient for this research. Because the area is to be continuously searched, there is no overall goal location. At each time step, the planner will decide the next couple of steps based on area uncertainty. A popular graphical approach for trajectory generation involves the use of Delaunay triangulation and Voronoi points, as described in [14]. In this method, each threat or area that is desired to be avoided is treated as a point. A unique circle is drawn between all triplets of threats and only the triplets whose circle does not enclose any other points are considered. These sets of points are referred to as the Delaunay triangulation, and the centers of the circles are known as the Voronoi points. Edges are then drawn between Voronoi points when their associated Delaunay triangles share an edge. The edges of this structure are all equidistant from the threats. These Voronoi cell structures then represent the safest path for the aircraft. Another graph method involves virtual forces. The eld can be thought of as consisting of springs and dampers. Each point connecting the lines of the UAV path is considered a spring, so the line can freely move about the unsafe areas, with the threats being analogous to dampers. If this system of springs, dampers, and masses is initialized at any conguration, it will evolve in time and reach a steady-state value according to the governing physics and converge to its potential energy minimum [14]. Physics based planners solve the steady state equations and generate a trajectory following the resulting line. An oine and online three dimensional path planner is provided in [15]. In this approach, path lines are represented by B-Spline curves. An evolutionary algorithm is used to generate the trajectory with the coordinates of the curve control points being the chromosome genes. The algorithm is a modied breeder genetic algorithm that incorporates some characteristics of classic genetic algorithms [15]. The algorithm was also extended to only use local information to achieve the goal location in a dynamic environment by utilizing potential elds to drive the aircraft in the direction of the destination.
9 2.2 Path Planning Path planningconsists of of waypoints (WP) to an aircraft. Way sending a set points are mapped reference points in the UAVs environment, usually consisting of such information as latitude, and altitude. research ignores the actual trajectory longitude, This control laws, although smart implementations can help generation low-level or autopilot alleviate certain problems. The accepted control architecturefor UAVs divides the control loops into generally a innerloop and outer loop. The inner control loop involves the an two levels, low-level actual controlling of the aircraftscontrol surfaces response to actuator inputs follow or in to loop consists of track a given point ortrajectory. The outer control thepath planning and decision makingbased on environment information and receiving feedback the onboard via mentioned, project performs sensors. As in this it is assumed that the autopilot the inner loop functionality. This paper will address various methods towards achieving the desired goals of the control loop. This control is illustratedin Figure which was outer loop 2.1, taken from [16].
Guidance Controller
Li
Vi Learning strategies, storage of information about environment
Di
Decision-making for where to search, what to engage, how to evade Pi
v i
2.1: Outer Loop Control Figure
2.2.1
Serpentine Patterns
Because the mission is a wide area search, many applications suggest the use of an oset serpentine pattern for each UAV. This is sometimes referred to as a lawnmower pattern. There are many dierent strategies to developing this pattern. The simplest method, which was utilized during the rst phase of this project, involved direct geometric calculations depending on the size of the search area, the desired aircraft separation, and the
10 number of aircraft. The rst version of this software did not take into account replanning for events such as loss of aircraft and could not plan around obstacles or no y zones (NFZ). A neural network approach to complete coverage path planning where the dynamic neural activity landscape represents the dynamically changing environment is provided in [17]. A multiple UAV coordination approach involving decomposing the search area into various polygons and assigning a UAV to each polygon while attempting to limit the number of turns is provided in [18]. Although applied to ground vehicle coordination, a complete coverage planner involving both polygon decomposition and alternating row allocation with collision avoidance is given in [19]. A method to generate paths using only local communications is mentioned in [20].
2.2.2
Graph Based Methods

Many of the approaches taken involve dividing the map into a discrete number of
grids. Each UAV can then move in discrete steps from one grid intersection to another. In the majority of the literature, the search area is divided into either square of hexagonal grids for simplicity. Possible path choices could then be crudely narrowed down by the aircraft control laws, such as maximum turn rate and turn angle. As the number of paths looked at ahead of time increases, the number of possible choices grows exponentially. Once all possible paths are generated, the branches are weighted, and various algorithms are utilized to nd the lowest cost path. This approach was followed and much of this research was taken from ideas in the following papers. One approach researched involved dividing the search area into square grids. The aircraft heading was limited to eight directions. At each timestep, the UAV could only choose three of the eight headings based on current direction. This limited the UAV turns to 45 . In order to search eciently, each agent predicted the state of the search area in the near future via feed-forward neural networks trained by a reinforcement learning algorithm. A centralized approach in which all agents shared the same neural network, a decentralized approach where each agent had its own neural network, and a greedy search algorithm was compared in [11]. This method was based on assigning a certainty value to each grid that measured the degree that it was searched. Another strategy dividing the search area into the square grids and limiting the UAV to the same discrete headings, treated all uncertain areas as possible destinations to
11 identify targets [21]. In this research, along with [16], a q-steps-ahead path planning method is used in which the UAV species q points ahead and replans every timestep. A certainty map is again used describing the degree to which the grid has been searched, driving the UAVs to search unexplored areas. The certainty map changes with each timestep by a dened function, capturing the notion of diminishing returns for subsequent searching of a grid. In order to achieve cooperative path planning, paths are rewarded based on target conrmation, environment exploration, thread avoidance, and cooperation. After the possible paths are generated, weights could be assigned to each branch and the problem becomes one of determining the lowest cost path. Various AI graph search algorithms could be utilized to nd the optimal solution such as Dijkstras and A*, or any other greedy and breadth-rst searches.
2.2.3
Stigmergic Approaches
Numerous methods of controlling swarms of UAVs, with the goal similar to this
project, are inspired by physical and biological phenomenon. Although there are numerous methods and algorithms used, a fundamental concept remains the same. This concept is known as stigmergy. Stigmergy is used to describe a broad class of multi-agent coordination mechanisms that rely on information exchange through a shared environment [3]. This again stresses the importance of the environment to these reactive systems and the coupling between the aircraft and the environment. 2.2.3.1 Pheromone-Based Approaches Emergent behavior is the property of high level complex behaviors emerging from simple low level rules. It is best described by the process of ant path planning to nd food, as described in [8]. As ants leave the nest, they follow scent markers known as pheromones. They choose each step from a uniform distribution over all possible directions, weighing more heavily along the path with the strongest scent. When they are carrying food, they drop scent markers known as pheromones and return to the nest. The key is that the pheromones evaporate over time, meaning that paths to depleted food sources eventually disappear. This process shows how simple rules for each ant lead to a global behavior of nding food for the nest. The key concept to emergent behavior is in complex strategies arising from sim-
12 ple rules. Accoring to [22], the attributes to swarming UAVs are simplicity, robustness, scalability, and economy. These principles are generally self-satised with pheromone-based cooperative control. This approach is criticized because it is at odds with the traditional centralized top-down system design. The main problem with top-down approach is that the usual predened order prevents adaptation [23]. Some theory and results relating increasing entropy leaks at the individual agent level to decreasing entropy at the system level, implying stability, are provided in [24] and [23]. In the pheromone-based method, the search area is divided into equally sized grids. Each UAV shares a map of the environment. As the UAVs traverse, they drop pheromones on each grid and communicate locally with other UAVs. Because the information content and maps are updated infrequently, low bandwidth communications are sucient [25]. During each timestep, the gradients of all pheromones are calculated, and the UAVs plan the path towards the resulting gradient. This naturally causes dispersion and eliminates search area redundancy. A complete description of digital pheromones, strengths of individual pheromones, and the fundamental pheromone propagation equations are provided in [25]. The performance of the algorithm is then compared to a xed pattern search for surveillance, target acquisition, and target tracking missions. It also proves that the developed algorithm behaves with repeatable performance. This algorithm was demonstrated onboard actual hardware to prove the concept. Another analysis of swarming algorithms utilizing pheromones is given in [7]. 2.2.3.2 Physics-Based Approaches Another popular approach is a physics based swarm control approach. It involves the use of potential elds as in electrostatics. The electric eld E(r) at a point in space is dened as the force felt by a unit charge at that point. The potential eld is dened as 21 =
P2 P1
E dr, or as a gradient E = . This is generated by the physical distribution
of charges according to Coulombs law. This law has been used to control swarm UAVs as described in [12]. The potential eld approach involves dropping charges as opposed to pheromones in various search locations and calculating the resultant electric eld. The aircraft then traverses the gradient of the resultant vector. Another physics-based approach involves the use of dynamic co-elds, which is another swarm behavior using localized
13 decision processes to generate emergent, highly sophisticated maneuvers [26]. The main dierence between physics and pheromone based approaches is that pheromone elds are quantitative while potential elds are qualitative. Although research has been done to reduce the power requirements, the continous eld approach generally involves more computation to determine the resultant potential eld vector. In a way, pheromones and potential elds share the same ideas. Markers are placed in the environment to attract or repel agents, resulting in paths that follow the calculated gradients. 2.2.3.3 Other Approaches There are other approaches also used to imitate simple biological behavior. One approach involves the use of a Gibbs sampler to determine the movement of nodes, modeling the swarm as a Markov random eld [27]. Another approach involves the use of ocking that consists of three simple steering behaviors: separation, alignment, and cohesion [28]. This approach has been previously used for intelligent ocking behavior in animation and is currently being researched for practical robot control use. Finally, nite state machines have been used to control multiple UAVs. The disadvantage to this method is that it is assumed that all states will be known. Finite state methods are discussed in [3] and [29].
2.3
Task Allocation
Countless methods exist for performing UAV task allocation, which is the assigning
of UAVs to tasks given resource constraints. Most methods involve assigning a cost function to possible UAV paths or tasks and determining the lowest cost solution. An approach using mixed-integer linear programming (MILP) to allow exibility to include more constraints is discussed in [4]. In this paper, the problem is formulated as a multi-dimensional multiplechoice knapsack problem (MMKP) in which an element must be chosen from each of multiple sets. Task allocation could also be handled by various network optimization models. A genetic algorithm for task allocation in cooperative control is provided in [30] in the context of destroying targets while minimizing ight time. Another major distributed task allocation scheme involves the use of economics and market mechanisms. A market approach to multirobot coordination involving maximizing prots with a dened revenue/cost model and allowing subgroup formations to improve eciency in a dynamic environment is described in [31]. The contract-net protocol [32] is
14 predominant in market mechanisms. In the classic market approach, an agent can auction its tasks to other agents who bid on the task with the associated cost. The task is assigned to the lowest cost agent. An equity coecient is introduced in [33] to balance each agents workload with respect to the other agents. Various auction methods related to agent theory are provided in [9]. English and Dutch auction approaches and their major dierences are outlined in [5]. Some algorithms naturally handle task allocation, such as the pheromone and potential eld approaches. For example, because these methods involve placing attractive and repulsive markers in the environment, markers of extremely high value could be placed on a target, and the closest UAV would naturally follow the strong gradient eld towards the marker. Descriptions of various types of pheromones used that depend on the mission is provided in [25].
15
Chapter 3
Investigation
3.1 Algorithm Overview
The operational requirements for developing the algorithm are rst provided, followed by system requirements, operation, and a detailed description of each component of the cooperative control framework. The major tasks of the algorithm involved creating a swarm network message protocol, devising a scheme for storing the information obtained from environmental events, allocating targets to identication aircraft, and dynamically assigning the search and ID aircraft ight plans. These tasks demonstrate the basic concepts of a reactive decentralized cooperative control algorithm, illustrating a high level of autonomy.
3.1.1
3.1.1.1
Operational Requirements
Previous Requirements The cooperative behavior algorithm was developed based on certain concepts of
operation. It built upon and improved previous versions of the algorithm, which consisted of centralized software that planned serpentine (lawnmower) type sweeping search patterns to traverse a search area, as described in section 2.2.1. An example of the lawnmower search pattern is illustrated in Figure 3.1, which is a screen capture of the simulation path planning environment from the previously developed algorithm. The sweeping search pattern provided a simple yet eective means of guaranteeing complete search area coverage. The previous operational concept was to have two types of aircraft, search and
16
Figure 3.1: Sweeping Search Pattern Setup identication (ID). The search aircraft would have a low resolution, wide eld of view (FOV) sensor for maximum coverage, and the ID aircraft would be equipped with a high resolution sensor to identify recognized targets. The search aircraft would traverse the lawnmower pattern and send information to the ID aircraft when targets were discovered. The ID aircraft would remain in a predened orbiting pattern until they received notication of a target location, at which time they would allocate the task autonomously and service the target, which demonstrated heterogeneous entities cooperating. This planning scheme had limitations, such as the complexity of planning this type of search pattern with the addition of dierent priority regions. Although the search pattern could be dynamically recalculated in the case of a loss of aircraft or other environmental events, it lacked exibility when the area had obstacles such as no-y zones (NFZ). Although methods have been formulated to solve the optimal sweeping pattern in case of obstacles, as described in [17] using neural networks and in [18] using polygon decompostion, it was decided that the strategy did not provide the desired capabilities. After the initial algorithm was developed as a proof of concept, various ideas were formulated. It was decided that the ID aircraft should also participate in searching the
17 area until a target was found. This would provide two major advantages over the initial algorithm. First, it would maximize resources by having the ID aircraft search while not tasked to a target instead of simply orbiting. Second, it would illustrate a higher level of autonomy by having a single aircraft serve multiple functions. Identical software would be installed on each aircraft, independant of the function, eliminating the need for dierent versions. The tasks would again be autonomously allocated without operator intervention. Continuous replanning of the lawnmower search pattern would have to be performed when targets were discovered so the ID aircraft could break from the search and after the targets were serviced so they could re-enter the search This could result in areas that would either be missed or overlapped, eliminating the eectiveness of the optimal complete coverage method. 3.1.1.2 Current Requirements The requirements for the new algorithm reected a similar concept of operation, but it addressed the limitations of the previous lawnmower search pattern. A basic wide area search scenario was again used as the basis for the requirements. This was a realistic intelligence, surveillance, and reconnaissance (ISR) mission that would prove the concept of the UAV swarm. The initial software would provide a foundation to build upon. Once the algorithm was successfully tested, modications and additions could be made to handle more complex scenarios. One major revision to the original algorithm addressing the dynamic replanning limitation was to utilize a more random search technique. In a realistic search environment, it could prove benecial to have a more unpredictable search pattern with deterministic overall results, which by denition implied emergent behavior. For instance, a partially random path planning function with weighted branches would provide an unpredictable path for each aircraft but the overall goal of having near complete search area coverage should be accomplished and repeatable. A major advantage of a more stochastic algorithm over a preplanned pattern like the serpentine pattern is that adversaries are less likely to evade the search [25]. A graphical search tree approach, similar to the method discussed in section 2.2.2 taken from [21], was proposed and is described later. Another major requirement was to have a decentralized algorithm. All planning and tasks were to be assigned via the onboard payload computer without a central processing
18 station. This required a dened set of messages to be sent to and from all nodes of the swarm. It was also undesirable to have any sort of master/slave architecture. All of the various auction and bidding algorithms, as discussed in section 2.3, contain a temporary auctioneer, or master. The algorithm was to eliminate the need for any central processing, whether it was only for a very short amount of time. This would allow the ground station or any other node to be completely removed from the system and the onboard software would continue to dynamically assign tasks and ight plans, providing a more robust design and eliminating any single point vulnerability. The system was also to be fully autonomous. This meant that all tasks were to be allocated and search paths planned without operator intervention. As discussed, a major benet of swarming behavior is that it elimates the necessity for an operator to perform low-level UAV control. The goal was to have an operator and a user, where the operator would control the entire swarm by sending high-level commands such as dening a search area or priority regions and the user would concentrate on the information being sent from the onboard sensors. The operator could also choose to monitor health status of individual aircraft of the swarm. This would prove a major step in UAV control, eliminating the manpower necessary for current systems. The search area was dened to be a rectangular area with two dierent priority levels. Figure 3.2 illustrates the specied search area for demonstration and test purposes. The red inner ring represents higher priority grids, which should be searched a dened number of times more frequently than the low priority blue grids. This area could represent a mission in which the perimeter is searched more frequently than the inside. Any rectangular shape could be chosen for the search area, and any grid could be chosen as priority for the algorithm. The heterogeneous swarm consisted of the same two types of aircraft, search and identication (ID), with the same functionality, target recognition and identication. The major dierence was that all aircraft would participate in searching the area of interest (AOI) until a target was discovered, at which time the optimal ID aircraft would be appropriately tasked to service (identify) the target and send an image to the user ground station. Once all active targets were serviced, all tasked ID aircraft would revert back to searching. Having multiple functioning aircraft would demonstrate the required high levels of autonomy and cooperation between the swarm. A simulation application had to also be developed to simulate automatic target
19
Low Priority
High Priority
Figure 3.2: Demonstration Search Area recognition (ATR). The simulation computer would act as another node operating over the payload communications link and send messages to the aircraft as they would y over a target. The target locations would be known and input into the simulation system before the mission was executed. The software would simply listen to all aircraft telemetry messages and determine if an aircraft was ying over a target grid. If so, the simulation computer would send a message notifying the aircraft of the target. The simulation environment would eventually be replaced by onboard ATR. It was also required that targets be revisited at a certain specied rate. This meant that a timestamp be placed on each identied target. The algorithm was to compare the current time with each target and if the maximum timeout value was reached, the target would be reset to an active state. The appropriate aircraft would then be assigned to the target and the timer would be reset again. This demonstrated the continuous monitoring of specied target locations. Another specication was that the software should dynamically handle the addition and loss of aircraft. This required that the swarm be aware of all active network connections and eliminate aircraft that were predicted lost. Proper initialization messages had to also be developed in order to increase awareness when various entities were added to the network.
20 This requirement was naturally met by the software initialization process. As opposed to the lawnmower search pattern, the aircraft in a random search would not have to recongure in case of aircraft loss, and because all data is contained onboard each aircraft, no information would be lost. 3.1.1.3 Summary To summarize, the following requirements were used in designing the software: 1. Autonomous Decentralized Cooperative Algorithm 2. Ability to Handle Priority Regions 3. Specied Target Revisit Rates 4. ID Aircraft Participating in Search These requirements led to certain software implications. In order to have a fully autonomous and decentralized system, processing had to be performed onboard each individual aircraft and data received from the swarm network had to be synchronized to ensure all solutions and calculations were consistent. Priority regions and random search patterns led to the search area being divided into grids and a look-ahead tree search used to determine paths. This required that the environment be stored onboard each aircraft, including target locations, search area grid status, and swarm location, eliminating the need to request the data from a central ground station or database. Revisit rates required the addition of timestamps to each grid as they were searched and when a specied timeout was reached the target would be reset from a searched to an active state so that it could be reassigned. Because the identication aircraft would also participate in the search, this aected grid sizes of the search area division due to the smaller footprint of the high-resolution sensors.
3.1.2
System Requirements
The developed cooperative behavior algorithm runs on any Microsoft Windows R
compatible computer. It was successfully tested on a computer running a variation of the Microsoft Windows XP R operating system (OS). Standard C/C++ function calls were used except when specic functionality was necessary, such as enumerating network card connections. All of the platform dependent calls were completely modularized, allowing
21 the system to be platform independent with minor modications. The specic classes that utilize the dependent Microsoft Win32 API are the low-level networking classes, the XML le processing classes, and the lockable object classes, although most of the Winsock networking calls follow the BSD socket standard. The only requirement is that the Microsoft MSXML Software Development Kit (SDK) or runtime libraries be installed. This library contains the necessary les to help parse the XML initialization le.
3.1.3
Architecture
The developed system consists of various components that receive input from the
network, avionics, and onboard sensor. The main algorithm consists of the following modules: 1. Network 2. Environment 3. Sensor Interface 4. Avionics Interface 5. Task Allocation 6. Path Planning Each of the components is further broken down to provide a modular system, in which specic blocks can be replaced when improving or expanding the algorithm in future versions. The system package diagram is illustrated in Figure 3.3. This diagram shows how the main cooperative behavior application interacts with all of the developed assistant libraries. As mentioned previously, this allows specic packages to be used by alternative algorithms without details of the packages. The developed swarm algorithm is a multi-threaded event based system, with dierent messages indicating various events. As events of interest occur, a specied callback function could be called to alert the main application. The registered functions are called as the appropriate information is received from the network, avionics, or sensor. All modules that utilize callback functions are totally reusable and operate transparent to the main application. This allows the main application to simply register a function and handle the
22
<<Network Library>>
<<Sensor Interface Library>>
<<Swarm Network>>
Cooperative Control Application
<<Environment>>
<<Planning>>
<<Avionics Interface Library>>
<<Utilities>>
<<XML Library>>
Figure 3.3: System Package Diagram
23 data appropriately as the function is automatically called, regardless of how the data is received. The algorithm is a completely modular system, as illustrated by the class diagram in Figure 3.4. Various components could be improved, replaced, or added in future versions if necessary. The diagram illustrates how the main application interacts with the developed classes and libraries. The individual modules are summarized below and discussed in detail in the appropriate sections. Related classes were grouped accordingly into packages. A layered architecture was chosen in which various layers or modules could be reused to test or integrate other algorithms in the future or compare performance. Other algorithms could use the appropriate classes or create custom classes to provide specic functionality. For example, if an algorithm required a totally dierent data storage mechanism but needed to receive messages from the swarm network, the networking libraries could be used and the environment classes customized or new ones developed. The inner details of the networking software would not have to be known. The interface could be used, a callback function registered, and data would be automatically received saving development time. All interface library parent classes are generic abstract classes that dene the required methods that must be overridden by the child classes. This allows the main application to use the generic parent class and instantiate the specic child class at runtime through polymorphism. The current algorithm utilizes this functionality by reading an initialization le, determining the appropriate interface, and casting the generic parent class to the specic interface class. It also provides a method of easily adding new components. For example, if a new avionics was used in the future, a child class could be created that implemented the dened functions through the parent class. The basic network library was developed and its functionality was inherited by the swarm network classes. The sensor and avionics interface provides a means of information exchange between the onboard avionics and sensor hardware/ATR software and the main algorithm. The XML library was developed as a wrapper to the Microsoft MSXML software library, to allow easy reusability of common XML le parsing functions. The environment class contains the swarm, search area, and target classes. Information is updated as it is received from the environment. The utility classes provide functions such as converting between geographic and grid coordinates. The main cooperative behavior modules of the algorithm are in the planning portion, which contains the path planning and task allocation
24
<<Network Library>>
<<Sensor Interface Library>>
CConnection
NIKON_Plugin.dll
CClient
CServer
CNetwork
CSensorInterface
SensorAPI.dll
Cooperative Behavior Swarm Network

CSwarmClient CSwarmServer
Environment
CEnvironment CTarget
CSwarmNetwork
CPacketManager
CAircraft
CSearchArea
Main Application
Planning
CSearchPathPlanner CGeoCalculator CTaskAllocation
Utilities
CGridCalculator CSwarmXmlParser
<<Avionics Interface Library>>
<<XML Library>>
CPiccoloInterface
CAvionicsInterface
MSXML2.dll
CXmlParser
Figure 3.4: Algorithm Class Diagram
25 classes.
3.1.4
Overall System Operation

The overall system concept is illustrated in Figure 3.5. There will be two personnel
communicating with the aircraft, each performing a dierent task. The pilot monitors the status of the swarm and the mission with aeronautical and aircraft status information. The user will have access to the sensory information from onboard the swarm and will make various requests to service certain areas and targets.
Figure 3.5: Overall System Operation The pilot information will initially be sent through the existing autopilot control link, because this information is already sent to the existing ground station software. Large amounts of detailed information is sent over this link, which is useful for debugging and safety purposes, but will no longer be needed once the system is tested and the existing OI software is eliminated from the loop. The link will eventually be replaced with the payload link and only crucial information necessary to successfully monitoring and controlling the swarm will be sent. This idea of separating command and control is prevalent in military
26 systems. The cooperative behavior software will reside on the payload computer, as depicted in Figure 3.5. The software accesses the environment through three interfaces and stores all necessary information in the environment database to access and perform calculations. Although it is depicted as a database, the environment is implemented as a class containing vectors of aircraft, targets, and search areas. The sensor interface is a software module that was integrated in order to access the sensor, which was a digital camera, through the payload computer. The ATR software will eventually use this module and provide inputs into the cooperative behavior software. Currently, the only sensor functionality utilized by the software is the changing of resolution and taking pictures. The avionics interface was developed and integrated to access the necessary avionics information for certain calculations and sending updated ight plans to the ight management system. A class was developed to wrap the communications SDK provided by the avionics manufacture. The abstract avionics interface was overridden to provide specic functions to send and receive messages to the autopilot. Additional classes would have to be developed if other avionics units were integrated. The nal interface that was developed was the communications interface. This module includes a set of helper functions to send and receive information over the payload link. The software uses simple TCP/IP communications and all routing and hopping is handled internally by the mesh network cards. This allowed a simple client/server model to be developed. Once integrated, these three modules provided an interface that is transparent to the type of hardware onboard the aircraft. Communications equipment, sensors, and avionics units could eventually be interchanged without aecting the overall operation of the cooperative behavior algorithm. Additional software classes would have to be developed to support various hardware added in the future. A high level state diagram is provided in Figure 3.6 to illustrate all of the basic states of the swarm software. The initialization state is where all mission parameters and user dened software variables are read from an XML le. The sensor interface is also initialized and an attempt is made to establish a connection with the ground station and simulation computer. During initialization, both the avionics and network interface threads are started and callback functions registered to begin receiving messages to update the
27 environment. Finally, the software enters the planning state, which constantly monitors the environment and sends either search or identication ight plans and allocates tasks. As depicted, all three portions of the algorithm must interact with the environment. To prevent simultaneous reading and writing of data, the application had to lock the environment before writing and unlock if when writing is complete. A more thorough description of each of the major components is provided in subsequent sections.
Begin Mission / Start Software
Initializing
Initialization Complete, Messaging Threads Started
Planning
Info Received Request Info
Avionics Interface Thread Avionics Messaging
If Telemetry, Waypoint Change, or Autopilot Mode Change Msg / Update Env
Network Interface Thread

Msg Parsed / Update Env
Environment
Network Messaging
Env Updated With Avionics Info Target Information: 1. Target Location 2. Target Type 3. Timestamps SWARM Information: 1. SWARM Telemetry 2. SWARM Tasks 3. Timestamps Network Information: 1. All Node Connections World Information: 1. Area of Interest 2. Priority Regions 3. Grid Status
Env Updated With Network Info
Figure 3.6: Overall System State Diagram
28
3.2
Software Initialization
An XML le was used to store initial conditions and specic mission setup pa-
rameters. This initialization le was advantageous because certain information about the mission could be recovered in case of power failure and parameters could be specied without recompiling the software. This le also allowed the search area to be hard coded until the ability to specify areas of interest was implemented into the user ground station. The XML le is used to setup necessary mission parameters, such as aircraft and sensor information, in order for the algorithm to successfully process events. This le is read on software execution, and the values stored in the environment. This provides a method to reinitialize the aircraft in case of power loss or software reboot. An example XML le, MissionParameters.xml, is provided in Figure 3.7 and illustrates the relevant elements.
Figure 3.7: Sample XML Software Initialization File The root element is <SWARM_INIT>. The <AC_TYPE> element contains all necessary aircraft information, such as sensor type, footprint, aircraft function, and aircraft ID. The <SEARCH_AREA> element contains all information about the mission AOI. In the future, this would be dynamically assigned as requests from the user are approved by the pilot. The ground station and simulation computer IP addresses are also given. Once the mission begins, the software will be started and enter the initialization
29 state, which is expanded and illustrated in Figure 3.8. The initialization state is where all system parameters are set and both the avionics and network threads are started. The rst state is where the XML initialization le is parsed. All aircraft parameters are read, such as aircraft and swarm identication number, sensor information, and avionics type. The aircraft function is also read from the le, which is currently either SEARCH or ID. After the aircraft information is parsed, mission parameters are read, such as the four corner points of the rectangular search area, the grid size, and the mission ight altitude. All of the information read from the le is then validated and the application exits if any errors occur. All known network entities and congurations are then read from the initialization le. This currently only consists of the ground user station and the simulation computer IP addresses. An attempt is made to establish a connection with both the ground station and the simulation computer. All swarm connections are persistent to eliminate the overhead of handshaking associated with TCP. The simulation computer is currently necessary to receive target messages and handle new aircraft messages, which are described below. After the connections are established, the sensor interface is initialized based on the sensor type eld of the XML le. The sensor interface class then loads the appropriate software modules. The sensor control modules are contained within dynamic link libraries (DLL). Currently, only a digital camera is supported. As new sensors are integrated, new libraries would be developed and loaded at runtime. For the tests, a Nikon digital camera was used. Next, the swarm server is started. The swarm server class inherits all of the basic TCP/IP server functions from the developed network library. Once started, the server registers a callback function and creates a new thread to listen for messages from the payload link. The main application registers callback functions for each of the dened network messages. All network messaging is handled by the networking classes and is transparent to the main application. The thread loops until messages are received. They are parsed and the appropriate callback function is called based on the message type eld. The conguration of the swarm network is currently handled by the simulation computer. In the future, it would be integrated into the user station. First, a new aircraft message is sent to the simulation computer, which stores all active aircraft connections. The simulation computer then iterates through all other aircraft and sends them notication of the new aircraft. Information regarding the existing aircraft on the network is sent to the
30
XML Initialization File: 1. AC Info (Search or ID, Avionics, ID) 2. Sensor Info (FOV, Type) 3. Mission Parameters (Search Area, Altitude) 4. Network Info (Ground Station IP, Sim IP) Begin Mission / Start Software
Initializing State Parsing Initialization File

Fields Checked For Validity / Exit On Error
Adding Network Clients

Ground Station And Simulation PC Connections Initialized
Env / Aircraft Info Configred And Stored
Storing Information
Initializing Sensor Interface
Sensor Interface Initialized
Starting Swarm Server

Swarm Server Started, Callback Functions Registered
Starting Avionics Interface Thread
New Aircraft (ACFT) Msg Sent To Notify Swarm
Sending New Aircraft Msg
Avionics Interface Initialized, Callback Functions Registered
Planning
Info Received
Request Info

Environment
Network Messaging
Env Updated With Avionics Info
Figure 3.8: Software Initialization State Diagram
31 new aircraft. This provides an easy mechanism for conguring the swarm. Although the network conguration is currently centralized and requires a ground station to keep track of all aircraft, it only occurs during initialization. This method was the easiest and most straightforward way to send aircraft notications. Another method that was developed but not implemented was to use functionality available in the network class. This class provides Microsoft Windows R specic functions to enumerate all connections associated with a specic network card. The IP addresses of all aircraft on the same network could be obtained. This would eliminate the need for ground processing. The new aircraft could send a notication message to all other aircraft and wait for replies. The functions were successfully demonstrated on desktop computers used during testing. Finally, the avionics interface thread is started based on the avionics type eld of the XML le. Currently, only the a single avionics unit is supported. The aircraft ID eld, which is the avionics serial number, is used to set up the serial connection (RS-232) with the avionics. Callback functions are then registered to alert the main application of telemetry, waypoint change, and manual/autopilot control messages. Once the connection is established, a thread is created to constantly poll the avionics message queue for new data. When new data is available, the appropriate callback function is called and the information passed to the main application. The software initialization process is probably the most important part of the swarm algorithm. It attempts to establish all required connections and interfaces. If any portion of the process fails, the application exits. Future versions could implement default values and generic interfaces so that the software will continue to work on minimum information in case of problems. Error recovery schemes and default interfaces could be implemented to provide a more robust fault tolerant initialization process.
3.3
Network
The communications, or networking, module performs all messaging within the
system. As information is received from the avionics or sensor, it is packaged and sent to all other aircraft in the swarm. Messages are also received from the swarm communication network, originating from other aircraft, the pilot, or the user. The following message sets were necessary for the successful operation of the swarm, based on the algorithm requirements and proposed implementation.
32 1. New Aircraft (ACFT ) 2. Telemetry (TEL) 3. Target (TGT ) 4. Path (PATH ) 5. Area of Interest (AOI ) 6. Image (IMG) Each message consists of a message ID, source, and data. As messages are received, the environment stored on each individual aircraft is updated appropriately and action is taken when determined necessary. The ACFT message is sent on initialization when a new aircraft enters the swarm. It consists of the aircraft conguration and the IP address. The conguration contains all information from the sending aircrafts initialization le, including aircraft type, aircraft id, function, sensor type, etc. This information is stored in the swarm portion of the environment database and a connection is established between the two aircraft. The conguration information is utilized to determine the functionality of each aircraft when tasks must be assigned. The TEL message consists of aircraft telemetry information, such as latitude, longitude, altitude, airspeed, and heading. When information is received from the avionics, a TEL message is formed with the appropriate data and sent to the swarm. When a TEL message is received from the network, the environment is updated with the new aircraft information. Currently, no other action is taken when this message is received. In the future, it could be utilized for collision avoidance or nearest neighbor calculations when optimizing the algorithm. These messages are necessary to ensure that all aircraft calculations result in consistent solutions, such as when tasking the optimal aircraft to service a particular target. The TGT message consists of target location (latitude, longitude), classication, and priority. In the future, ATR software will interface with the onboard sensor and send a ag if a target is recognized, and the information would then be formed into a TGT message and sent to the swarm. Currently, this message is received from a ground ATR simulation computer, which is discussed later. If the aircraft is an ID aircraft and a TGT message is
33 received, the task allocation and path planning modules are called and the target is serviced accordingly, based on lowest cost. The PATH message is used to keep the swarm environment search area grids updated. As new search paths are calculated, the grid points in the path are sent to the swarm. Once a new search path is formed and sent, the aircraft must nish ying all points of the path before it can be re-tasked. This commitment prevents aircraft from updating grids as searched when receiving PATH messages when an aircraft may not have searched the grid before being diverted to a dierent task. When the search mission begins, an AOI message is sent describing the search area. A new search area could also be requested by the user in the future. When the area is requested, the pilot would approve/disapprove based on aeronautical information such as ight regulations. The message consists of all latitude and longitude points of the polygon. The current implementation requires a rectangular search area for simplicity. It is read from the initialization le and the appropriate callback function is called to simulate receiving the AOI message from the network. These four points are used to divide the search area into equally sized grids proportional to the sensor footprints, as provided ini the XML initialization le. A network library was developed to handle all swarm network communications, utilizing standard client/server architecture over TCP/IP. The server uses a select() based socket polling method as opposed to creating a separate thread for each connection or entity of the swarm. The networking components were designed to be completely reusable. An additional layer was added to the basic client/server modules where a callback function could be specied, which would allow the networking functions to be transparent to an application. The application could register the callback function and let the networking modules call the function when messages were received. If desired, the basic client/server modules could also be used to send generic data through the TCP/IP network. The network class diagram is shown below in Figure 3.9. The CConnection base class provides basic socket connection functionality shared between both clients and servers. The CNetwork class provides helpful networking functions, such as determining a network card IP address and enumerating through all other nodes of the network. The developed algorithm includes a CSwarmNetwork class that utilizes and completely wraps the developed network library, containing both a swarm client and server, which are children of the basic client and server classes. The CSwarmServer class provides
34
<<Network Library>>
CConnection
CClient
CServer
CNetwork
Swarm Network
CSwarmClient CSwarmServer
CSwarmNetwork
CPacketManager
Figure 3.9: Network Communications Class Diagram the additional functionality of allowing callback functions to be registered on the following message events: 1. New Aircraft 2. Telemetry 3. Target 4. Path 5. AOI 6. Unknown The CSwarmClient class provides a resend counter. After a specied number of sends, the connection is assumed to be terminated, which could indicate loss of aircraft. If the aircraft would come back online, it would reinitialize and send another new aircraft message.
35 This class also contains the type of aircraft, allowing specic messages to only be sent to particular types of aircraft. When testing other algorithms, a function could be registered and appropriately called when messages are received from the swarm network. The network messaging thread runs simultaneously with the planning state of the main algorithm. Figure 3.10 illustrates the network messaging state. The swarm server loops continuously until a message is received from the payload communications link. After a message is received it is parsed and the appropriate callback function is called based on the message type eld. The main application takes dierent actions depending on the type of messages received. When a new aircraft message is received, the elds are parsed and a connection is established with the aircraft. The new aircraft network connection is also added to the environment. The new aircraft will be sent all future telemetry messages from every aircraft of the swarm. The specic avionics that was utilized for the hardware-in-the-loop (HIL) test updated every second (1Hz). This rate was reasonable since the tests were conducted with four aircraft. In the case of a path message, the grid timestamps and search count elds are updated in the environment. When a telemetry message is received, the appropriate aircraft telemetry is updated in the environment. Currently, a history of three telemetry packets is stored. This number could be adjusted in future versions if more information is needed. In the case of a new area of interest message, all parameters are calculated. These parameters include the number of equally sized grids, the search area rotation angle with respect to north (azimuth), and the center point of grid zero. Target messages are slightly more complex. The rst step is to check the source of the target message. If the source of the message originated from the simulation computer, which currently takes place of onboard ATR, the aircraft would send the target message to all ID aircraft of the swarm. If the aircraft that received the message is an ID aircraft, it is assumed that the target has been serviced by the high resolution sensor, so the image is sent to ground. If the source of the message is another aircraft of the swarm, the software enters the task allocation state. All active targets are then optimally assigned to the ID aircraft. If any targets are assigned to the aircraft that received the target message, the environment is updated with the target list. This process is repeated until the application exits. The planning portion of the algorithm relies on environmental information that is updated from network messages. If the communications link fails, no tasks will be assigned and no information will be up-
36
Initializing
Planning
Info Received
Request Info Basic Messages: 1. New Aircraft (ACFT) 2. Telemetry (TEL) 3. Target (TGT) 4. Path (PATH) 5. Area of Interest (AOI) 6. Priority Region (PR) 7. Image (IMG)
Environment

Network Messaging State Update Env
Listening
Msg Received
Env Updated
Updating Environment
Case TEL Msg Case PATH Msg Case AOI Msg
Grids Updated
Parsing Message
Case TGT Msg
Update Grids
Rotation Angle, Grid Width/Height, Grid 0 Center Point Calculated
New Aircraft Reply Sent
Calculate Parameters Check Source

If Source = Search Aircraft If Source = Simulation PC (Future ATR), Send To All ID Aircraft Case ACFT Msg Aircraft Added To Network
Initializing Connection
Tasks Allocated
Sending Reply
Allocating Tasks
If I Am An ID Aircraft, Send Image To Ground
Setting Target List

Target Lists Set
Sending Message
Sending Image
Image Sent
Figure 3.10: Network Messaging State Diagram
37 dated. This would result in the aircraft only performing search, although the search would be inecient due to no information on areas already covered by other aircraft. The communications is vital to the success of the algorithm. As a reactive system, a feedback loop from the environment must continually be present in order to process information.
3.4
Avionics Interface
The avionics interface is a generic interface to the hardware allowing access to air-
craft telemetry information and sending new ight plans. Data is received at a certain rate, coded into a TEL message, and sent to the swarm to be stored onboard in the environment. When new ight plans are generated, they are sent to the avionics through this interface. It is important to note that the swarm algorithm does not perform any control of the aircraft. It only dynamically allocates tasks and generates new ight plans to complete the tasks by dening all waypoints of the path. Once the waypoints are sent to the ight management system (FMS) of the avionics, it is assumed that the aircraft will follow the specied route. An abstract base class, CAvionicsInterface, denes an interface that all child classes must implement. The main application contains a pointer to this class. When the initialization le is parsed and the avionics type is determined, the pointer is cast to the appropriate child class through polymorphism. This architecture allows specic classes to be developed for each avionics that is integrated. The overall interface denition and function calls will remain the same, as dened in the interface, but the implementation will dier. The avionics messaging state, illustrated in Figure 3.11, is similar to the network messaging state. It runs in parallel with the planning state of the main application. Once the connection is initialized, the thread loops continuously looking for new avionics data. When new information is available, the appropriate callback function is called and the environment is updated. Special action is taken when a telemetry message is received from the avionics. Besides the environment being updated, the dened network packet is formed and sent to all other aircraft of the swarm. The CAvionicsInterface class provides the functionality of allowing callback functions to be registered on the following message events: 1. Telemetry
38
Initializing
Planning
Info Received Request Info

Environment
Network Messaging
Update Env
Avionics Interface Thread

Avionics Messaging State Env Updated With Network Info
Listening
Packet Received
Env Updated
Updating Environment
TEL Msg Sent To Swarm Case Telemetry Packet
Parsing Packet
Sending Message
Figure 3.11: Avionics Messaging State Diagram
39 2. Waypoint Change 3. Autopilot (AP) Mode Change When telemetry is updated from the avionics, the thread calls the registered callback function with all of the data. The second function that can be registered provides notication when the autopilot tracked waypoint changes. This allows the software to keep track of what waypoint of the ight plan is currently being traversed. Although it is not utilized in this software, the AP mode change function alerts the main application via callback when the autopilot is switched between manual control and auto mode and vice versa.
3.5
Environment
Data storage is an important aspect of the swarm algorithm. Information about
the environment, such as aircraft telemetry, search area grids, targets, etc. has to be stored. A le indexing scheme was proposed for dynamic mission information storage as opposed to running a full database engine on the payload computer, which would add unnecessary overhead and processing power. If additional storage capability was required in the future, such as relational les, the method provides easy expansion. The proposed le indexing scheme is described below for all environment types of storage: search areas, targets, and swarm information. Utilizing data les to store information provided a robust design in case of any type of failure. If the software had to be rebooted or there was a system power failure, the data could easily be retrieved and the system restored to a previous state. This also eliminated numerous initialization request messages, allowing minimal exchange of information over the network with a small cost of additional onboard storage. The following sections describe the structures used to store information for this project, although any metric could easily be added if deemed necessary. The environment class is where all swarm information is stored. The class diagram is shown in Figure 3.12. The CEnvironment class is a container to store all aircraft, target, and search area information. Because of the callback messaging architecture of the algorithm, precautions had to be taken when developing the environment. Multiple threads could try to access the environment simultaneously, which could lead to unpredictable results. The Win32 API lockable objects were used to solve this problem. All classes of the
40 environment are based on a lockable parent class, providing both a lock and unlock method. When the main algorithm accesses any portion of the environment, care must be taken to rst lock it and then unlock it when the operation is complete.
Environment
CEnvironment
CTarget
CAircraft
CSearchArea
Figure 3.12: Environment Class Diagram
3.5.1
Search Area Information

The search area, which is currently dened to be a rectangular region with polygon
priority areas, is completely described by the four corner points. Specifying all four points allows the search area to be rotated in any orientation with respect to due north. The search area would rst be divided into square grids as illustrated in Figure 3.13 when a new AOI message was received. The large block divisions would each be stored in a separate data le. The name of the le would represent the specic block location with respect to the search area. For example, the le 001002.dat would represent block(1, 2) of the search area. Each le would be limited to a size of about 0.5 M B so the information could easily be loaded and unloaded dynamically to and from memory. The latitude and longitude ranges of the grids that each block covered could be calculated, allowing each aircraft to load the appropriate le based on the current ight region. Each block, represented by a le, would be further divided into grids of Xf t by Y f t. The dimensions X and Y could be related to the smallest aircraft sensor crosssection. A preliminary value of 90% of the sensor cross-section was chosen. (Cross-section = the diagonal of the rectangular sensor footprint.) This method would provide adequate coverage of a section independent of aircraft heading and minimal bank angle across the center coordinate. Each of the individual grids would be represented by a programming structure in the le, such as the sample shown below. The search area would consist of an
41
Search Area (Lat1, Lon1) 001001.dat

2 2
001002.dat (X + Y ) = 90% of (Sensor Footprint) min X ft.
Y ft.
Each block represents an information structure inside the grid file
(Lat2, Lon2) 002001.dat 002002.dat
Figure 3.13: Search Area Storage array of the individual grids. struct IndividualGrid { short int nIndexX; short int nIndexY; short int nSearchCnt; bool bSearched; short int nPriority; double dTimestamp; }
// // // // // //
X Coordinate of Grid Y Coordinate of Grid Number of Times Grid Has Been Searched Has the Grid Been Searched Integer Priority Level of the Grid Last Time that the Grid was Searched
The grid structure described contains 17 bytes of information. With a 0.5 M B le limit, each le could contain (0.5 106 )/17 29, 412 grids. The number of grids in each row and column, assuming a square search area, is equal to the square root of the number of grids, i.e. 29, 412 171.5. An integer number of 171 grids in each row and column could be used, allowing storage of 29, 241 grid structures per le. The nal total memory of each le would be approximately 497.1 KB. The total search area dimensions that each le
42 covered could be calculated by the following formulas if the minimum rectangular sensor footprint cross-section, SensorF Pmin , is known: Y = 0.9 SensorF Pmin sin(45 ) X = 0.9 SensorF Pmin cos(45 ) SearchArea = X Y For example, if the minimum sensor footprint cross-section was 100f t and the footprint covered a square area, the resultant grid size would be X = Y 63.64f t. The total search area would be approximately (171 63.64)2 = 10882.442 square feet or 2 square miles. A single le would be sucient to cover the tentative demonstration search area. Any desired metric describing the search area grids could be added and stored. For instance, a pathname string representing the image of the grid could be stored and used to access images from each grid. The structure representing the grid corresponding to the current sensor footprint location could be loaded from the appropriate le with simple pointer arithmetic, as illustrated in the following pseudo-code: Algorithm 1 Obtaining Correct Grid Structure dist = getDistanceBetweenP oints(Lat1, Lon1, Lat2, Lon2); X = dist cos( ); Y = dist sin( ); [GridX, GridY ] = getGridCoordinates(X, Y ); Index = GridX + (GridY XCnt); ptr = (ptr + Index sizeof (struct SearchT able));
WHERE:
// Lat1, Lon1 = Upper Left Corner of File Coordinates (Search Area) // Lat2, Lon2 = Aircraft Position // XCnt = Number of Grids per Row (171) Figure 3.14 illustrates the dierent values and variables in the pseudo-code. As each grid is searched, the appropriate structure is updated and the path sent to the other aircraft in the swarm, who in turn update their tables. All aircraft would then contain
43 a consistent representation of the environment status assuming ideal network communications.

X=dist*cos(Ango) (Lat1,Lon1) Ang Y=dist*sin(Ango)
Figure 3.14: Search Area Calculations
di st
(Lat2,Lon2)
(GridX,GridY) = (4,3)
3.5.2
Target Information
As targets are recognized by the ATR software, the aircraft would be notied and
a message containing the necessary information would be sent to the swarm. Tasks would optimally be assigned to the appropriate ID aircraft. As TGT messages are received, a new target object would be instantiated and added to the environment. The environment consists of a vector of targets. class CTarget : public CLockableObject { public: CTarget(unsigned int nGrid, unsigned short int nType, double dTime); ~CTarget(void); . . . private: unsigned short int m_nTgtType; unsigned int m_nGrid; double m_dTimestamp; };
// Target Type // Target Grid // Timestamp Of Search
44 The classication (target type) eld would provide information about the type of target (red or blue forces). Currently, the possible target types are UNKNOWN, HOSTILE, FRIENDLY, and NEUTRAL. It is assumed that the ATR software would eventually provide the target location information and the identication software would determine the target classication and priority. In the future, a target priority eld could be added and used to service important or high threat level targets immediately. The timestamp eld provides a mechanism to incorporate the required revisit rates. During each iteration of the main algorithm, the timestamp is compared to the current system clock and if the time expired the target would be reset to an active state.
3.5.3
Swarm Information
When a message is received from the swarm, the specied aircraft information is
updated onboard in the environment container class. Each aircraft of the swarm is stored as a map data type, which allowed a key to index the vector of aircraft objects. The aircraft integer serial number was used as a key to allow easy access to each aircraft as messages were received. The following class was used to store information about each aircraft in the swarm: Telemetry information received from the avionics would be packaged into the specied format and sent to the swarm. This information could be utilized when computing costs for all ID aircraft to allocate a particular task. It could also be necessary if collision avoidance or other improvements were implemented in the future. The history of each aircraft of the swarm could be limited to a particular timeframe or number of instances. More recent messages could eliminate and overwrite outdated information, resulting in a minimal amount of necessary storage.
3.6
Path Planning
The path planning module consists of both the search plans and the identication
ight paths when targets are assigned. The search path planning module of the algorithm would constantly generate and update the aircraft ight plan. A dynamic search method was implemented to address priority regions and revisit rates and to make the individual aircraft search plans somewhat undeterministic. This method also allowed both search and identication aircraft to participate in the search, utilizing all resources, until targets were
45
class CAircraft : public CLockableObject { public: CAircraft(int nID); // CAircraft Constructor ~CAircraft(void); // CAircraft Destructor . . . private: int m_nAcftID; int m_nSwarmID; AircraftCfg_t m_sAcftCfg; std::string m_strAcftIP; double m_dTimestamp; bool m_bTelemetryRx; bool m_bManualFlag;
// // // // // // //
Aircraft ID Swarm ID Aircraft Configuration Aircraft IP Address Telemetry Data Timestamp Telemetry Has Been Received Manual Mode // // // // Telemetry History Current Target List Current Target New Target List
std::deque<Telemetry_t> m_qHistory; std::vector<int> m_vTgtList; int m_nCurrentTgt; bool m_bNewTgtListFlag;
Direction m_eCurDir; // Current Grid Direction int m_nCurGrid; // Current Grid Number int m_nCurWP; // Current Waypoint };
46 assigned. Once a target was recognized, the appropriate ID aircraft could then be diverted from search to service the target. Both portions of the ight planning module are discussed below. Figure 3.15 illustrates the planning state. This is the main state of the cooperative behavior application. After software initialization is complete, the rst ight plan is initialized, adjusted to meet avionics limitations, converted to geo-coordinates, and sent to the avionics. The major eort was in determining the rst point of the ight plan. This required rst determining the current aircraft heading, grid location, and whether it is already inside the search area or not. Special attention had to be given if the aircraft would fall into a potential deadlock grid, which would cause the search path planner to fail. Pre-turning, which is an adjustable parameter that commands the autopilot to start executing a turn to track the next waypoint before ying over the current waypoint, was used. The rst point had to be a certain distance away so the aircraft could properly align its course for the rst point. The aircraft was then commanded to track the rst waypoint of the ight plan. The main application loops continuously until an exit condition occurs. First, all necessary information is retrieved from the environment. This includes aircraft telemetry, average GPS groundspeed, and elapsed time since the last telemetry message was received. Next, all targets are checked for timeout. If any target needs to be revisited, it is set to an active state and the target message function is called, which handles the task allocation. The aircraft target list is then retrieved from the environment. This determines whether search or identication paths will be planned. High-level pseudo-code for the algorithm is provided below in algorithm 2. This illustrates the overall system operation and cooperative behavior decision making process. If there are no targets available the aircraft continues to search the area. If targets are available, the appropriate identication path is planned. The other states included in the planning state are independent of the aircraft function. The distance to the next waypoint of the ight plan is estimated. This was necessary to eliminate errors in taking pictures over the center point of the grid resulting from incorrect aircraft location data because the avionics only sends telemetry information once a second. The worst case scenario would be that the most recent position data was received just under a second ago, which could result in close to a 100f t error with the velocity of the aircraft used for testing. The estimated distance was then compared to the picture waypoint. If the picture grid was missed, the current picture waypoint was incremented. If the aircraft was within
47
Initializing
Initialization Complete, Messaging Threads Started Planning State
Initialize FP Adjust FP
Flight Plan Safety Adjustments Flight Plan Initialized
ID Path Planned
Plan ID Path
Search Path Updated
Plan Search Path
Send Image Convert FP

Converted From Grid To Geo Coords Image Sent To Ground Image Stored
If Target Grid
Take Picture
If Center Of Grid If Missed Waypoint
Send FP
Flight Plan Sent To Avionics Aircraft Sent To Waypoint
Update WP
Waypoint Updated Retrieved Env Info
Estimate Distances
No New Targets
If Last Waypoint In Plan
Send To WP
Retrieve Information
Check Revisits
Expired Targets Updated
If ID Aircraft And New Targets
Check Target List
Request Info
Info Received

Environment
Network Messaging
Figure 3.15: Planning State Diagram
48 Algorithm 2 Cooperative Control Algorithm Main Loop Start Avionics Interface Thread Start Server Thread Register Callback Functions Initialize Sensor Interface while Mission in Progress do Check for Target Timeouts if Searching then if Target List > 0 then Set Function to Identication end if Estimate New Position Calculate Distance to Next Grid Center Point if Grid Distance < Take Picture end if if Grid N umber = Last Grid of P lan then Load Grid Information Generate Possible Path Tree Calculate Weights and Costs Determine Minimum Cost Path Send Flight Plan to Avionics end if else if Identication then Get Target List Determine Flight Plan Send Flight Plan to Avionics while T arget List > 0 do Get Telemetry Estimate New Position Calculate Distance to Target Grid Center Point if Grid Distance < Take Picture Send Target Picture to Ground Station end if end while end if Set Function to Searching end while then then
49
Algorithm 3 Avionics Interface Thread while Mission in Progress do Receive Telemetry Information Form SWARM Message Update Environment SWARM Send SWARM Message to Nodes end while
Algorithm 4 Swarm Server Thread Receive Message if ACFT Message then Add New Aircraft to Network Send Aircraft Information Reply else if TEL Message then Update Environment Aircraft Telemetry else if TGT Message then if Search Aircraft then Update Environment TGT else if ID Aircraft then Update Environment TGT Allocate Tasks end if else if AOI Message then Divide Area into Grids Update Environment Search Area else if PATH Message then Update Environment Search Area Grid Status end if
50 a certain threshold of the grid center point, a picture was taken. The picture was sent to the user ground station if the grid contained a target. If the aircraft was within a certain distance to the last waypoint of the search ight plan, the next search path was calculated and sent to the avionics.
3.6.1
Search Path Planning

While performing search, the aircraft would assign a specied number of path
points at a time. The points would correspond to the center of the individual grids discussed in the environment search area information section. When ight plans were determined, they would be sent to the swarm so the search area status could be updated. Variables describing the number of points to look ahead when planning would be specied. Figure 3.16 illustrates the possible aircraft headings and an example of the possible paths an aircraft could traverse when looking ahead two grids at a time. The possible search paths form a traditional search tree. In the algorithm implementation with square grids, there would be eight possible headings to traverse between the midpoints of the grids. This number could be reduced to three possible headings assuming that the aircraft was limited to 45 turns between grids because of limiting ight dynamics. This crudely takes aircraft turn radius into account without the need for complex aircraft dynamic calculation. The current aircraft heading would determine the search tree of possible paths. Each path of the search tree was weighted by a cost function, and the path with the overall lowest cost would be chosen and updated. The rst step in the algorithm would be to obtain the heading of the aircraft. Once the heading was known, the search tree would be generated with all possible paths. The grid structures representing all points of the possible path would be loaded into memory, so the search count and priority values could easily be retrieved, which play an important role in the cost function. The weights of all possible paths of the search tree would be summed, and the path with the minimum overall weight chosen. A sample search path cost function, which weighs individual branches of the search tree, is given below. It could include variables to favor high priority grids and areas that have not been searched or any other desired metric based on mission requirements. BranchW eight = k1 T imestamp + k2 P riority W eighttotal = BranchW eighti
51
Figure 3.16: Search Area Path Planning The variables represent a tradeo between searching unknown areas, high priority areas, and areas that have not been searched for an extended period of time, indicating higher uncertainty. The Timestamp variable would represent the time elapsed since a particular grid has been searched. In future versions, various uncertainty levels could be implemented. The priority variable represents a priority level of the specic grid, providing a means of searching high priority areas more often. The search tree contains a total of 3r W eighttotal values, assuming there are three possible headings at each grid, where r is the number of grids ahead in which the algorithm searches. For example, if looking ahead two grids at a time, there would be 32 = 9 permutations and therefore total weight values. The minimum of all the weight values would be chosen and the ight plan updated and sent to the swarm. Planning search paths involved searching a graph consisting of all possible aircraft directions and nding the path of lowest cost. Initially, an articial intelligence search class library written by Peter Bouthoorn and available from the CMU Articial Intelligence Repository was researched. The class library simplied general graph searching algorithms. It included classes to search uniform cost graphs and nd the lowest cost path. After testing and integration, it was determined that the software was not suitable for our search path planning algorithm due to the following reasons. A graph had to rst be stored by recursively iterating through all possible paths and costs for the aircraft. The
52 algorithm then found the lowest cost path to a specic goal. For the swarm search purpose, a specic goal does not exist. Each branch must be searched and the goal becomes the node with the lowest cost. Also, the search algorithm recursively iterates through each branch of the graph to determine the lowest cost path. It was realized that there was no reason to iterate through the graph twice, once to nd all possible paths and then to nd the lowest cost path. As the graph was searched for all possible paths, the costs were simultaneously stored saving both time and computational eort. The search path planning state diagram is illustrated in Figure 3.17. The application enters this state when either the last waypoint of the search ight plan has been reached or all targets have been serviced. First, the current aircraft location and heading is retrieved from the environment. All possible branches of the search tree are then recursively expanded one at a time. The current grid position was rst updated and then checked for deadlocks. If the grid was a deadlock grid, the branch was eliminated from the search, otherwise the cost was calculated. If the cost was greater than the current minimum cost, the branch was eliminated to save computation time. If the cost was less than the minimum cost and the search tree depth was equal to the specied number of plan-ahead points the minimum cost path was stored. After all branches were searched the best path was returned and sent to the avionics.
3.6.2
Identication Path Planning

When a task is allocated to a particular aircraft, a ight plan would be generated
and sent to the avionics. Currently, a straight line path from the identication aircraft position to the target would be generated, ensuring the target is serviced in the simplest and quickest manner. A straight line path is also used in calculating the distance parameter of the task allocation cost function. In future versions, the identication path planning module could be improved to rst generate a path that takes into account such information as restricted y zones and obstacles and then calculate the distance of the generated path using trajectory generation methods discussed in section 2.1. A sample ID path plan is illustrated in Figure 3.18. The rst point of the ight plan would be assigned to a location a specied distance from the current aircraft position along the same heading. All other points would be the center point of the grid containing the assigned targets. Once all assigned targets have been serviced, the aircraft would begin
53
If Last Waypoint In Search Plan Or All ID Tasks Complete Plan Search Path State
Entire Tree Searched / Return Best Path
Expand Branch
Branch Recursively Expanded
Update Parameters
Grid Position And Heading Updated Branch Pruned
Check Deadlocks
Grid Not Deadlock
Deadlock Grid
Eliminate Branch
Minimum Cost Path Stored
Cost > Min Cost
Calculate Costs
Cost <= Min Cost
Store Path
Check Depth
Depth = Plan Ahead Points
Cost Function Includes: 1. Grid Priorities 2. Grid Timestamps 3. Grid Search Counts
Figure 3.17: Search Path Planning State Diagram
54 searching the area again until other targets are found.
TGT1 UAV
X ft
TGT2
Figure 3.18: Identication Path Planning Figure 3.19 illustrates the identication path planning process once targets were assigned. First, the target list was retrieved from the environment. The target coordinates are then converted to grid numbers and the waypoints are set to the center points of each target grid. In order to improve performance, the rst waypoint is set to be 100m away from the aircraft along the same heading path.
3.7
Task Allocation
If designated as an ID aircraft and a TGT message is received, a cost function
is used to calculate the cost to service the target. The developed algorithm is capable of assigning multiple targets to inidividual UAVs. Currently, each aircraft calculates and knows the cost of all ID aircraft in the swarm. Although having each aircraft performing calculations of all other ID aircraft is not optimal and may not scale to a large swarm, it could be improved in the future. After the calculations are performed, which are assumed to be synchronized between all aircraft due to each aircraft receiving all messages and storing the information in the environment, any assigned targets are serviced. When targets are recognized, a particular ID aircraft must be tasked with servicing the target. The task allocation would be performed by evaluating a cost function and tasking the aircraft with the lowest overall cost. The variables considered when evaluating the cost function currently consist of the total distance from the target, the value/priority of the target, and information regarding aircraft dynamics. In future versions, other parameters aecting the total cost to service a target could be included, such as wind. Because the task
55
Case TGT Msg And Source = Search Aircraft Plan ID Path State
Retrieve Target List

Target Tasks Retrieved
Calculate Grids
Target Grids Calculated
Initialize WP
First Waypoint Initialized Waypoint Added To Flight Plan
Add Waypoint
All Target Tasks Assigned
Figure 3.19: Identication Path Planning State Diagram
56 allocation cost function is isolated, the calculations and parameters can be altered without aected the rest of the software.
3.7.1
Description
The following pseudo-code demonstrates the basic task allocation algorithm.
Algorithm 5 Task Allocation Get Active Targets from Environment Get ID Aircraft Information while Number Active Targets > 0 do Find Lowest Cost Target for Each Aircraft Keep Overall Lowest Cost Assign Target to Aircraft Update Positions Add Cost to Total Aircraft Cost Delete Target from List end while An example environment and possible steps resulting from the algorithm are illustrated in Figure 3.20. First the initial conditions are given. Next, the lowest cost target for each aircraft is determined. The target with the absolute lowest cost is assigned to the appropriate aircraft, the aircraft position and total cost is updated, and the target is deleted from the list. This process is repeated until all targets have been assigned. Although this simplied example only takes into account distance between aircraft target, the general algorithm and idea applies to any cost function. The example would result in aircraft #1 servicing the top two targets and aircraft #2 servicing the bottom target. Figure 3.21 illustrates the task allocation state diagram of the algorithm. A description of the task allocation and example scenario is given in the theoretical algorithm section. If an aircraft message is received from an aircraft, task allocation is performed. First, all active targets and ID aircraft are retrieved from the environment. Aircraft positions are then initialized and costs are calculated and sorted. The aircraft with the overall lowest cost will be assigned the appropriate target. The position will be updated and the target deleted from the target list. This process is repeated until all targets have been
57
d1
d2 d 1
d3
2 Aircraft 1 2 Total Cost Environment Initial Conditions, 0 0 AC Costs = 0 Aircraft 1 2
2 Total Cost Aircraft 1 Cost = d1, 3 dT = d1 Aircraft 2 Cost = di i=1 0
1
d1
d1
d2 d 1
d3 d 1 d2 d3
2 Aircraft 1 2 Total Cost Aircraft 1 Cost = d +d , T 1 dT = dT+d1 3 Aircraft 2 Cost = di 0

i=1
2 Aircraft 1 2 Total Cost Aircraft 1 Cost = d +d1, T dT 3 Aircraft 2 Cost = di d +d +d

1 2 3 i=1
Figure 3.20: Task Allocation Illustration
58 assigned. The tasks assigned to the calling aircraft are then kept and stored in the environment.
3.7.2
Cost Function
A sample equation that could be used to model the cost of a particular aircraft to
service a target is given below: T otalCost = Dist + ((Hdg Azimuth)/180) T R The Dist variable represents the distance between the aircraft and target and the Hdg variable is the current aircraft heading. The Azimuth variable represents the angle between the aircraft and target location with respect to due north, and TR represents the aircraft turn radius. The maximum dierence between the aircraft heading and azimuth would be 180 , which would indicate that the aircraft is traveling in the exact opposite direction of the target. The dierence is divided by 180 to weigh and add the aircraft turn radius to the total cost. A heavier weight, implying a larger cost, would be added as the aircraft heading becomes further from the azimuth between the aircraft and target. A weight value could also be used to dierentiate between targets of various priorities. It would be a parameter that represents a tradeo between either performing tasks on high priority targets in a timely manner or on the closest targets rst. It is assumed that the target value would be an input to the system, by either the ATR software or ground operator. When a large number of aircraft and/or targets need to be assigned, evaluating every permutation of the cost function for the entire swarm to nd the minimum value could be computationally infeasible. Instead, a genetic algorithm or other search mechanism could be used to allow multiple tasks to be assigned to any individual aircraft of the swarm in the future.
3.8
ATR Simulation Software

The ATR simulation application interface is illustrated in Figure 3.22. The soft-
ware was used to simulate future onboard ATR functionality. It allowed targets to be added, edited, or deleted. In order to add targets to the simulation environment, various elds had to be specied. The rst eld was the target classication eld, which could be hostile,
59
Case TGT Msg And Source = Search Aircraft Task Allocation State
Retrieve Active Targets

Active Targets Retrieved From Env
Retrieve ID Aircraft
ID Aircraft Retrieved From Env Cost Function Includes: 1. Target Distance 2. Aircraft Heading
Initialize Positions
Aircraft Grid Positions Calculated And Initialized
Calculate Costs
Costs Calculated for Entire SWARM
Sorting Costs
Costs Ordered
Low Cost Assigned Target Removed And Target List > 0
Update Aircraft Positions

Low Cost Aircraft Projected Position Updated
Remove Target
Target List = 0 (All Targets Assigned)
Figure 3.21: Task Allocation State Diagram
60 friendly, neutral, or unknown. The target coordinates would also have to be specied so that the simulation software could compare telemetry messages with target locations and send notications when the aircraft was in a target grid. An optional target description eld could also be specied to help distinguish various targets. Information is displayed on the selected target. The status eld indicates whether the target has already been sent to an aircraft or not. As the software performs various tasks, it would display status messages to the user.
61
(a) Target Editing
(b) New Target Setup
Figure 3.22: ATR Simulation Software Interface
62
Chapter 4
Summary and Conclusions

4.1 Algorithm Test Results
Various tests were conducted to verify that the software was operating as expected. Each of the individual modules were tested as they were developed followed by overall system integration/operation tests. The results of the search aircraft path planner and the task allocation function are discussed in this section. Although the algorithm consisted of many other software components, these two specic modules were the main portion of the algorithm so only the testing involving them are noted. They illustrate the proof of concept of the algorithm based on the dened requirements and demonstrate the high levels of autonomy intended.
4.1.1
Search Path Planning Tests

A simple container was developed to allow the search path planner to be tested
apart from the rest of the algorithm. The number of aircraft and steps to perform in the simulation had to be specied. The test container would iterate through all steps and save various metrics to a le. The application did not take actual distances and ight dynamics into account. For instance, an aircraft ying a diagonal path would have to travel a farther distance than an aircraft traveling a straight line (by a factor of 2), which in turn would lead to the aircraft planning at dierent times. In the testing implementation, at each computer time step all aircraft would sequentially calculate the next path. In evaluating the search path planner, a set of metrics had to be devised to prove
63 that the requirements were satised. The following outputs were used to evaluate the eectiveness of the search path planner: 1. Percent Covered 2. Area Histogram 3. Average Grid Coverage These metrics would verify that the software satised the requirements to cover a dened search area, while covering priority grids twice as often as the rest of the AOI. During each timestep, the number of grids searched was compared to the overall number of search area grids. This number represented the percent coverage (P C) of the search area. P C = 100 (GridsSearched /GridsT otal ) Optimally, this value would reach 100%, but it is not feasible to repeatedly have perfect coverage given the nature of the stochastic algorithm. The goal was to maximize the coverage and have the results eventually stabilize even with the presence of timeouts. The timeouts represent uncertainty in a searched grid as the timestamp becomes outdated. A histogram was used to analyze the eectiveness of the algorithm in covering priority grids twice as often as other grids. The average grid coverage parameter (AvgCoverage = (GridCount /GridsT otal )) was used to attempt to measure the certainty
that the grids would be covered equally. The average coverage of both regular and priority grids was measured. The standard deviation of the number that each grid was searched was also taken. Ideally, this number would be extremely small. Each grid should be covered approximately the same number of times. The rst tests involved determining the number of aircraft required to search a particular area with a specied timeout value. Sample results are plotted in Figure 4.1. This could be used as a tool to estimate the required number of aircraft to search a given area at a required coverage. Other tests involved the eectiveness of the algorithm when priority areas were specied. It was required that the priority areas be searched twice as often as regular grids. The number of times each grid was searched was saved to a le and the histogram plotted. An example plot is given in Figure 4.2. The table below the gure summarizes the results. This test demonstrated the eectiveness of the algorithm. Although it is random in
64
Figure 4.1: Percentage of Search Area Covered nature, the nal results were somewhat predictable. This illustrates the successful overall repeatable outcome of a random search from a set of simple rules. The table shows that the priority areas are indeed searched approximately twice as often as the other grids. This was for 500 iterations of the algorithm. The standard deviation increased with the number of iterations. Although this was recognized as undesirable, it was accepted and decided to be improved for future versions by further adjusting the cost function.
Average Standard Deviation Median
Non-Priority 34.5 10.97377 39
Priority 70.59524 13.08003 77
4.1.2
Complete System Tests

A testing station was set up utilizing all hardware components of the system,
providing a complete hardware in the loop (HIL) test platform. There were three swarm software test stations. The test stations consisted of a simulation PC to run a HIL avionics ight simulation which included aircraft dynamics. The simulation PC was connected to the avionics via USB. The payload computer was connected to the avionics via a standard
65
Figure 4.2: Search Area Grid Count with Priority Areas serial cable, allowing telemetry information to be received and ight plans sent directly to the ight management system (FMS) of the avionics. All cooperative behavior messages were sent over the swarm payload link. The avionics contained a separate link that was strictly used for command and control information via the ground station. The rst test station also included a digital camera connected to the payload computer via USB. Only one camera was used during testing. The third test station provided software debugging functionality. A standard laptop was used as opposed to the payload computers so standard programming tools could be used to step through the code and observe operation. The debugging laptop could also connect to all other computers remotely through UltraVNC. This application allowed easy le transfer to the payload computers after modications to the algorithm were made and errors xed. Complete hardware system testing was successfully conducted. Although the algorithm was not demonstrated in ight, the ground tests illustrated the eectiveness of the algorithm with hardware-in-the-loop simulation including ight dynamics. The only major limitation to conducting the tests on the ground was the ability to adequately model the communications link during ight over various distances and airspeeds, but the concentration of this eort was solely on the algorithm.
66
Figure 4.3: Full System Test Setup
67
Chapter 5
Recommendations
5.1 Future Work
The goal of the project was to successfully demonstrate a basic cooperative behavior algorithm that could be integrated onboard an operational UAV system. Once the basic foundation was developed, the algorithm would be continuously revised and improved to address all limitations and problems. Many of the modules could be built upon to continuously improve the system. The identication path plans could be drastically improved by taking into account aircraft dynamics. Currently, the waypoints of the ight plan are set over top of the center points of the search area grids. In order to improve the ight plan, the aircraft heading and bank angle would have to be taken into account. One method could be to take the line tangent to the turn radius circle traversing through the target and evenly space waypoints about the circle and line as illustrated in Figure 5.1. This would ensure that the proper turning is taking place and the nal straight line segment would ensure that the aircraft is close to level ight which would increase sensor accuracy. The cost functions could also be improved to include other variables when determining both the identication and search path plans. It was desired to have priority regions searched a specied number of times more frequently than regular areas. Testing results show that the standard deviation of the number of times each grid was searched increases with time. This implies that areas of high priority will be searched more often, but as time passes, less distinction would be made. Improvements in the cost function could help to decrease the standard deviation. Although the algorithm is random in nature, the overall
68
Figure 5.1: Improved Identication Path Plan search results should be as predictable as possible. ATR software would also replace the simulation computer in future releases. Instead of messages being sent from the simulation computer, a callback function could be registered to alert the underlying algorithm of targets. In the future, telemetry information could be utilized to route messages more eciently to nearest neighbors, act as input to a collision avoidance system, or calculate other aircraft costs to complete a particular task. The telemetry history could also be used to predict future aircraft actions. The area of interest (AOI) message should be sent from the user station in the future. After a certain area is requested and approved by the pilot, the coordinates of the polygon desired to be searched will be sent to each aircraft of the swarm. Appropriate ight plan recongurations will then occur, reecting the new search area.
69
References
[1] T. W. McLain, Coordinated control of unmanned air vehicles, Tech. Rep. ASC-992426, Air Vehicles Directorate of the Air Force Research Laboratory, 1999. [2] Z. W. Spritzer, Comparison of path-planning and search methods for cooperating unmanned aerial vehicles, master of science in aerospace engineering, West Virginia University, 2004. Masters Thesis. [3] H. V. D. Parunak, Making swarming happen, in Proceedings of Swarming and Network-Enabled C4ISR, January 2003. [4] J. Bellingham, M. Tillerson, A. Richards, and J. How, Multi-task allocation and path planning for cooperating uavs, in Cooperative Control: Models, Applications, and Algorithms, pp. 119, Conference on Coordination, Control, and Optimization, November 2001. [5] D. M. Hart and P. A. Craig-Hart, Reducing swarming theory to practice for uav control, in Proceedings of the 2004 IEEE Aerospace Conference, vol. 5, pp. 3050 3063, IEEE, March 2004. [6] O. of the Secretary of Defense (OSD), Unmanned aircraft systems (uas) roadmap, 2005-2030, 2005. [7] P. Gaudiano, B. Shargel, E. Bonabeau, and et al., Control of uav swarms: What the bugs can teach us, in 2nd AIAA Unmanned Unlimited Conference and Workshop and Exhibit, AIAA, September 2003. [8] H. V. D. Parunak, go to the ant: Engineering principles from natural multi-agent systems, in Annals of Operations Research, vol. 75, pp. 69101, Springer Netherlands, January 1997. [9] G. Weiss, ed., Multiagent Systems: A Modern Approach to Distributed Articial Intelligence. The MIT Press, 1999. [10] R. W. Beard and T. W. McLain, Multiple uav cooperative search under collision avoidance and limited range communication constraints, in Proceedings of the 42nd Conference on Decision and Control, vol. 1, pp. 2530, IEEE, December 2003.
70 [11] Y. Yang, M. M. Polycarpou, and A. A. Minai, Opportunistically cooperative neural learning in mobile agents, in Proceedings of the 2002 International Joint Conference on Neural Networks (IJCNN 02), vol. 3, pp. 26382643, May 2002. [12] H. V. D. Parunak, M. Purcell, and R. OConnell, Digital pheromones for autonomous coordination of swarming uavs, in AIAAs 1st Technical Conference and Workshop on Unmanned Aerospace Vehicles, Systems, and Operations, AIAA, May 2002. [13] R. W. Beard and V. Stepanyan, Information consensus in distributed multiple vehicle coordinated control, in Proceedings of the 42nd IEEE Conference on Decision and Control, vol. 2, pp. 20292034, December 2003. [14] S. A. Borto, Path planning for uavs, in Proceedings of the American Control Conference, vol. 1, pp. 364368, IEEE, September 2000. Visiting Scientist, AFRL / VAAD. [15] I. K. Nikolos, K. P. Valavanis, N. C. Tsourveloudis, and A. N. Kostaras, Evolutionary algorithm based oine/online path planner for uav navigation, IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, vol. 33, pp. 898912, December 2003. [16] K. Passino, M. Polycarpou, D. Jacques, M. Pachter, Y. Liu, Y. Yang, M. Flint, and M. Baum, Cooperative control for autonomous air vehicles, in Proceedings of the Cooperative Control Workshop, December 2000. [17] S. X. Yang and C. Luo, A neural network approach to complete coverage path planning, IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, vol. 34, pp. 718725, February 2004. [18] I. Maza and A. Ollero, Multiple uav cooperative searching operation using polygon area decomposition and ecient coverage algorithms, in 7th International Symposium on Distributed Autonomous Robotic Systems, June 2004. [19] G. D. Winward, T. M. Petro, S. A. Gray, and A. K. Rekow, Static and dynamic path planning for multiple ground vehicles performing cooperative missions, in AUVSIs Unmanned Systems North America 2005, AUVSI, June 2005. [20] J. Schlecht, K. Altenburg, B. M. Ahmed, and K. E. Nygard, Decentralized search by unmanned aerial vehicles using local communications, in Proceedings of the International Conference on Articial Intelligence, vol. 2, pp. 757762, 2003. [21] Y. Yang, A. A. Minai, and M. M. Polycarpou, Decentralized cooperative search by networked uavs in an uncertain environment, in Proceedings of the American Control Conference, vol. 6, pp. 55585563, IEEE, June-July 2004. [22] B. Clough, Swarming intelligence, in Swarming Network Enabled C4ISR Conference, pp. Section F, 25, January 2003. [23] O. Bogatyreva and A. Shillerov, Robot swarms in an uncertain world: Controllable adaptability, International Journal of Advanced Robotic Systems, vol. 2, pp. 187196, September 2005.
71 [24] H. V. D. Parunak and S. Brueckner, Entropy and self-organization in multi-agent systems, in Proceedings of the 5th International Conference on Autonomous Agents (AGENTS 01), pp. 124130, ACM, ACM Press, August 2001. [25] J. A. Sauter, R. Matthews, H. V. D. Parunak, and S. A. Brueckner, Performance of digital pheromones for swarming vehicle control, in Proceedings of the 4th International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 05), pp. 903910, ACM, ACM Press, July 2005. [26] R. Chalmers, D. Scheidt, T. Neigho, S. Witwicki, and R. Bamberger, Cooperating unmanned vehicles, in AIAA 1st Intelligent Systems Technical Conference, AIAA, September 2004. [27] J. Baras and X. Tan, Control of autonomous swarms using gibbs sampling, in Proceedings of the 43rd IEEE Conference on Decision and Control (CDC), vol. 5, pp. 4752 4757, IEEE, December 2004. [28] C. Reynolds, Boids: Background and update. [29] K. Altenburg, J. Schlecht, and K. Nygard, An agent-based simulation for modeling intelligent munitions, in Proceedings of the 2nd WSEAS International Conference on Simulation, Modeling, and Optimization, WSEAS, 2002. [30] G. Chen and J. Jose B. Cruz, Genetic algorithm for task allocation in uav cooperative control, August 2003. [31] M. B. Dias and A. Stentz, A market approach to multirobot coordination, Tech. Rep. CMU-RI-TR-01-26, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, August 2001. [32] L. Xu and H. Weigand, The evolution of the contract net protocol, in Proceedings of the 2nd International Conference on Advances in Web-Age Information Management (WAIM 01), pp. 257266, ACM, Springer-Verlag, 2001. [33] T. Lemaire, R. Alami, and S. Lacroix, A distributed task allocation scheme in multiuav context, in Proceedings of the International Conference on Robotics and Automation (ICRA 04, vol. 4, pp. 36223627, IEEE, April 26 - May 1 2004.

Control

Cargado por

Información del documento

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

Control

Cargado por

Copyright:

Formatos disponibles

A Decentralized Cooperative Control Framework for Multiple UAVs

Copyright 2007 Dustin Michael Geletko

Notation and Acronyms

control, group coordination, and onboard route replanning.

FIGURE 4.0-2. TREND IN UA AUTONOMY.

Figure 1.1: UAS Autonomy Levels PROCESSOR TECHNOLOGIES

Submitted to the 2003 IEEE Conference on Decision and Control

Cooperative Waypoint Path Planner J

2.1: Outer Loop Control Figure

Graph Based Methods

E dr, or as a gradient E = . This is generated by the physical distribution

<<Sensor Interface Library>>

Cooperative Control Application

<<Avionics Interface Library>>

Figure 3.3: System Package Diagram

<<Sensor Interface Library>>

Cooperative Behavior Swarm Network

<<Avionics Interface Library>>

Figure 3.4: Algorithm Class Diagram

Overall System Operation

Begin Mission / Start Software

Avionics Interface Thread Avionics Messaging

If Telemetry, Waypoint Change, or Autopilot Mode Change Msg / Update Env

Network Interface Thread

Env Updated With Network Info

Figure 3.6: Overall System State Diagram

Initializing State Parsing Initialization File

Adding Network Clients

Env / Aircraft Info Configred And Stored

Initializing Sensor Interface

Sensor Interface Initialized

Starting Swarm Server

Starting Avionics Interface Thread

New Aircraft (ACFT) Msg Sent To Notify Swarm

Sending New Aircraft Msg

Avionics Interface Initialized, Callback Functions Registered

Avionics Interface Thread Avionics Messaging

If Telemetry, Waypoint Change, or Autopilot Mode Change Msg / Update Env

Network Interface Thread

Env Updated With Avionics Info

Env Updated With Network Info

Figure 3.8: Software Initialization State Diagram

Begin Mission / Start Software

Avionics Interface Thread Avionics Messaging

If Telemetry, Waypoint Change, or Autopilot Mode Change Msg / Update Env

Env Updated With Avionics Info

Network Interface Thread

Case TEL Msg Case PATH Msg Case AOI Msg

Rotation Angle, Grid Width/Height, Grid 0 Center Point Calculated

New Aircraft Reply Sent

Calculate Parameters Check Source

Setting Target List

Figure 3.10: Network Messaging State Diagram

Begin Mission / Start Software

Network Interface Thread

Avionics Interface Thread

TEL Msg Sent To Swarm Case Telemetry Packet

Figure 3.11: Avionics Messaging State Diagram

Figure 3.12: Environment Class Diagram

Search Area Information

Search Area (Lat1, Lon1) 001001.dat

001002.dat (X + Y ) = 90% of (Sensor Footprint) min X ft.

Each block represents an information structure inside the grid file

(Lat2, Lon2) 002001.dat 002002.dat