Evaluation of Pilot and Quadcopter Performance from Open Loop Mission Oriented Flight Testing

Abstract

Ease of control, portability and efficiency in versatile applications have made Unmanned Aerial Vehicle (UAV) very popular. Considering various usefulness, safe operation of UAV is important and to ensure safe operation, proper synergy between pilot and UAV is mandatory. For this reason, individual evaluation of both pilot and UAV performance is vital so that pilot can accomplish a task with the assigned system without any accident. In this study, a new evaluation technique of pilot and UAV performance is presented based on flight test results of a mission task of following a desired path. Seven pilots are categorized into two groups based on their experience level and a quadcopter is categorized into three groups based on level of autonomy associated with it. Path error is calculated in time domain to distinguish between pilot levels and level of autonomy of UAV. Path error metrics show that novice pilots make more error than experienced pilots and error increases from more autonomous to less autonomous UAV. For frequency do main analysis, transfer function modeling is done including human operator in the open loop so that full scenario of the flight, from pilot to UAV can be analyzed. Frequency domain analysis helps to identify system complexity, stability and fastness based on level of autonomy as well as pilot performance based on experience level. Apart from time and frequency domain analysis, Cooper-Harper rating scale is used by the pilots to rate the UAV based on ease of control. Along with time and frequency domain variables, Cooper-Harper rating is included as predictors in the modeling of evaluation of pilot and quadcopter performance. The parameter estimation of regression model shows the change in model outcome for both pilot and UAV level with the variation of predictor values. In the end, a verification test case is included where an eighth pilot flies the same quadcopter to complete the same task and variables derived from the flight data of this single flight test are placed in the binary logistic regression model equation to predict pilot experience level and multinoial logistic regression model equation to predict UAV autonomy level. The established model can predict pilot experience level and UAV autonomy level correctly that matches with the real case. The evaluation technique developed in this thesis shows a path to evaluate pilot and quadcopter performance individually, that can be used to train pilots to accomplish a specific task with the assigned UAV system.