{'fraction_non_alphanumeric': 0.02698933921101165, 'fraction_numerical': 0.007242139354954793, 'mean_word_length': 4.601410934744268, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
In-flight range optimization of multicopters using multivariable extremum seeking with adaptive step size

Limited flight range is a common problem for multicopters. To alleviate this problem, we propose a method for finding the optimal speed and heading of a multicopter when flying a given path to achieve the longest flight range. Based on a novel multivariable extremum seeking controller with adaptive step size, the method (a) does not require any power consumption model of the vehicle, (b) can adapt to unknown disturbances, (c) can be executed online, and (d) converges faster than the standard extremum seeking controller with constant step size. We conducted indoor experiments to validate the effectiveness of this method under different payloads and initial conditions, and showed that it is able to converge more than 30% faster than the standard extremum seeking controller. This method is especially useful for applications such as package delivery, where the size and weight of the payload differ for different deliveries and the power consumption of the vehicle is hard to model.


I. INTRODUCTION
Multicopters are used in a wide range of applications such as aerial photography [1], transportation [2], search and rescue [3] and inspection [4], thanks to their simplicity in fabrication and control, high maneuverability and low cost. However, the limited flight range of most available platforms [5] severely constrains their range of applications.
One way to address the limited flight autonomy problem is novel hardware design. For example, [6] designed a triangular quadrotor, which has one large rotor for lift and three small rotors for control, combining the energy efficiency of the large rotor and the fast control response of the small rotors; [7] designed a hybrid quadcopter which is able to do both aerial and ground locomotion. When the vehicle operates in the ground locomotion mode, it only needs to overcome the rolling resistance and uses much less power compared to flying; [8] proposed a quadcopter capable of in-flight battery switching.
Algorithm-based optimization is another way to reduce multicopters' power consumption. By planning energyefficient trajectories or by implementing energy-aware control algorithms, this approach does not require changes on existing hardware and is thus economic to deploy. Algorithmic improvements can be achieved through model-based or model-free methods. Model-based methods (e.g. [9] and [10]) benefit from fully exploiting the capabilities of the system, but are dependent on the availability of an accurate power consumption model of the system. The vehicle's power consumption model is usually derived from analyzing  The proposed extremum seeking controller with adaptive step size can find the speed and heading that maximize the flight range of the vehicle when flying along a user-defined geometric path. Besides, it adapts to unknown disturbances, such as different payloads. Experiments were conducted to evaluate the performance of the proposed method, which can be seen in the video: https://youtu.be/xuT-eGATesA.
its electric and aerodynamic properties. For example, [11] and [12] introduced power consumption models of the battery, electric speed controller and motor, and [13] introduced the aerodynamic power consumption of the propeller based on momentum theory. Model-free methods (e.g. [14]), on the other hand, could take into account hard-to-model effects, such as changes in vehicle components' performance due to aging and changes in the aerodynamic drag of the vehicle due to carrying different payloads or different wind speed. Model-free methods are especially useful for applications such as package delivery, where the size and the weight of the package differ for different deliveries.
In this work, we present a model-free, online approach to find the speed and heading of a multicopter that maximizes its total flight distance (range) using a multivariable extremum seeking controller with adaptive step size. Path planning for multicopters typically has some additional (redundant) degrees of freedom (e.g. the speed and heading of the vehicle). The proposed method autonomously sets the reference vehicle speed and heading along a predefined geometric path (e.g. a path for delivering a package from the warehouse to its destination) and parameterizes it into a reference trajectory, which is tracked by the low-level controller of the vehicle. The extremum seeking control is a technique for finding the input to a system that achieves the optimal output, and a detailed description and stability analysis can be found at [15] and [16]. For example, to find the minimum-power flight speed of the vehicle, the extremum seeking controller applies a periodic perturbation to the reference speed. And by monitoring the corresponding changes in the vehicle's power consumption, it estimates the gradient of the vehicle's power with respect to speed and performs gradient descent to find the optimal speed.
Besides, unlike the standard extremum seeking controller, which does gradient descent with a constant step size, we propose a novel extremum seeking controller that adapts the step size based on the first and second moments of the gradient estimation. The idea comes from Adam [17], a popular stochastic optimization method for training neural networks. Compared to the standard multivariable extremum seeking controller, the proposed adaptive step size method can achieve faster convergence. In indoor experiments we show the effectiveness of the proposed method in finding the optimal speed and heading by comparing the results from the extremum seeking controller with ground truth data; we also show the performance improvements of this new method compared to the standard extremum seeking controller.
In summary, the main contributions of this work are: (a) the proposal of a novel extremum seeking controller with adaptive step size, which converges faster than the standard extremum seeking controller, by blending the ideas from the machine learning field, (b) an extension of our previous work [18] of finding the optimal range speed of multicopters by taking into account the optimization of the vehicle's heading, and (c) experimental validation of the proposed method and comparison with the standard extremum seeking controller.

II. MULTICOPTER MODELING
In this section we define the reference frames, briefly introduce the dynamics and a power consumption model of the quadcopter. The model is given to help to understand the effect of the vehicle's speed and sideslip on its power consumption. It is not used for the flight range optimization.

A. Reference frame definition
As shown in Fig. 2, two sets of reference frames are defined -an inertial frame I attached to the ground and a body-fixed frame B attached to the Center of Mass (COM) of the quadcopter.

B. Quadcopter dynamics
The quadcopter is modeled as a rigid body with six degrees of freedom: three degrees of freedom from the linear translation along the three axes of the inertial frame I and three degrees of freedom from the three-axis rotation from  the body frame B to the inertial frame I, described by an orthogonal rotation matrix R. Each propeller i (i = 1, ..., 4), produces a thrust f i = (0, 0, f i ) and a torque τ i = (0, 0, τ i ), both expressed in B.
According to [19], the drag force f d (expressed in I) is consisted of a linear term and a quadratic term where ν ∞ corresponds to the free-stream velocity expressed in I, and µ 1 and µ 2 represent the drag coefficients. In addition, we assume that the drag force acts on the center of mass of the multicopter and thus no torque is produced because of it. The orientation of the vehicle with respect to the freestream velocity ν ∞ is described by the angle of attack α and the sideslip β, as shown in Fig. 2. The angle of attack α is defined as the angle between ν ∞ and the plane given by 1 B and 2 B ; the sideslip is defined as the angle between 1 B and the projection of ν ∞ in the plane given by 1 B and 2 B .
With x and its derivatives denoting the translational position, velocity and acceleration of the vehicle, g denoting the gravity acceleration, all in the inertial frame I, the translational dynamics of the vehicle is given by In addition, with ω and its derivative denoting the angular velocity and acceleration, expressed in B, the rotational dynamics of the vehicle is given bẏ where S(ω) is the skew-symmetric matrix form of the vector cross product such that

C. Power consumption
Following [5], [20] and [14], we assume that the total power consumption p (measured at the terminals of the battery) is proportional to the aerodynamic induced power p induced [13] p = 1 η p induced (6) where η lumps the power losses from the battery to the propellers. In forward flight p induced is computed as where ν ∞ = ν ∞ 2 is the magnitude of the free-stream velocity and ν represents the induced velocity applied by the propeller to the surrounding air. The scalar κ is an empirical correction factor. The induced velocity ν is implicitly defined by where ρ is the density of the air and r is the radius of the propellers. A more detailed derivation and explanation of the power consumption model can be found in our previous work [18]. When the vehicle is flying at a low speed, increasing the flight speed can help to increase the free-stream velocity ν ∞ and to decrease the induced power consumption. This effect was also observed by [5] and [13]. When the speed is further increased, the drag force f d becomes significant and large motor thrusts are needed for the flight, which causes power consumption to increase. The sideslip also affects the vehicle's power consumption by affecting the drag force f d if the vehicle is not axisymmetric. Thus, by affecting the power consumption of the vehicle, the speed and sideslip affects the vehicle's range.

III. MULTIVARIABLE EXTREMUM SEEKING WITH ADAPTIVE STEP SIZE
In this section we introduce the multivariable extremum seeking controller which is able to find the optimal range speed and sideslip of a multicopter, given a user-defined geometric reference path. This method does not depend on any power consumption model of the vehicle and can adapt to unknown disturbances such as different payloads. Changes in the vehicle's speed and sideslip affect the power consumption and thus the range of the vehicle, which is experimentally demonstrated in Section IV.

A. Cost function derivation
In this section we derive the cost function used in flight range optimization. The cost function's derivation follows our previous paper [18]. We assume that the multicopter (a) has energy E ∈ [E empty , E full ] during the operation, such as energy stored in the battery and does not have inflight battery charging, (b) is moving with constant speed v := v 2 and sideslip β, and (c) is using constant electric power p.
The total flight distance d flight is defined as Thus, maximization of the flight range corresponds to minimization of p/v and p/v is selected as the cost function for the optimal range extremum seeking problem. Fig. 3: The block diagram of the adaptive step size multivariable extremum seeking controller. The goal of the controller is to find the optimal rs and rv to minimize the cost function q(rv, rs). The scalar r v0 and r s0 represent the plant's initial setpoints for speed and sideslip. The frequency of the highpass and low-pass filters is set, respectively, to ω hpv and ω lpv for speed and ω hps and ω lps for sideslip. The scalar kv and ks are related to the step size of the extremum seeking controller and both of them should be positive numbers to minimize the cost function q(rv, rs). The diagram of the standard extremum seeking controller does not have the step size adapter and the output of the low-pass filter directly goes to the integrator, while other parts of the diagram are exactly the same.
B. Adaptive step size multivariable extremum seeking controller The proposed adaptive step size multivariable extremum seeking controller is shown in Fig. 3. It can find an unknown vehicle flight speed r * v and sideslip r * s that minimize the cost function q(r v , r s ) without depending on any power consumption model. For the optimal range goal, the cost function is instantaneous power over speed. The derivation of the cost function can be found in Section III-A.
The extremum seeking controller approximates the gradient of the cost function and performs gradient descent to minimize the cost [16]. To get the cost function's gradient with respect to speed, the extremum seeking control uses a dithering signal a v sin(w v t), a demodulation signal sin(w v t), a high-pass filter HP F v and a low-pass filter LP F v to estimate the gradient of the cost function q at the current set point ∂q ∂v (r v0 +r v ). We assume that the set point r v0 +r v changes much slower than that of the dithering and demodulation signal. The Taylor series expansion of the cost function is where O(r v0 +r v ) stands for higher order terms. The highpass filter HP F v would suppress the low frequency signal q(r v0 +r v ), and thus the output of the high-pass filter, q hpv , is Multiplication with the demodulation signal results in which is defined as ξ v . It has a low frequency component (or average) given by where R = lim T →∞ 1 T ∞ 0 a v sin 2 (w v t)dt. Thus, the output of the low-pass filter, q lpv , is an estimate of the cost function's gradient with respect to speed. For the same reason, the output of the low-pass filter LP F s , q lps , is an estimate of the cost function's gradient with respect to sideslip.
The difference between the proposed extremum seeking controller and the standard multivariable extremum seeking controller is the step size adapter, whose pseudo-codes are shown in Algorithm 1. The idea comes from the adaptive moment estimation algorithm (Adam) [17], which is commonly used in the stochastic optimization of objective functions in machine learning, such as training neural networks.
Input q lp (q lpv or q lps ), Output g (g v or g s ) Define and initialize β 1 , β 2 , and threshold m 0 ← q lp0 ; (Initialize the first moment) v 0 ← q 2 lp0 ; (Initialize the second moment) The adapter takes in the output of the low-pass filter (q lpv for speed and q lps for sideslip), which is the gradient estimation and outputs g, which is passed to the integrator to perform gradient descent. The effective step size for gradient descent is k v g v /q lpv for the speed optimization and k s g s /q lps for the sideslip optimization, and the step size adapter changes them by setting g v and g s . By taking the exponential moving average of the low-pass filter output, it helps to reduce the noise in the gradient estimation. The first output from the low-pass filter q lp0 is used to initialize the moving average. By dividing the first moment with the square root of the second moment ( √ v i ), the output g of the adapter will be similar for the sideslip and the speed, leading to similar convergence speed for these two variables if k v and k s are set the same; in addition, it outputs a small value when the gradient estimate has a large uncertainty (m i is small) and vice versa, which is good for the stability of the extremum seeking controller. When the square root of the second moment √ v i is less than the threshold defined, which means the extremum seeking controller is close to convergence, the output g will decrease as √ v i decreases to avoid oscillations ofr v andr s near convergence.
For the multivariable extremum seeking controller to work for both the speed and the sideslip simultaneously, the frequency of their disturbances ω v and ω s should be distinct [16]. One choice of the parameters is detailed in the experimental evaluation section (Section IV).
The role of the extremum seeking controller in the vehicle control architecture is shown in Fig. 4. Given a desired geometric path, instantaneous power measurements and instantaneous speed and sideslip measurements, the extremum seeking controller sets the reference tangential speed and sideslip along the desired path and parameterizes it into the reference trajectory, which is tracked by the low-level position and attitude controller. The power measurements come from an onboard voltage and current sensor, and the speed and sideslip measurements come from a state estimator based on a motion capture system.

IV. EXPERIMENTAL EVALUATION
Indoor experiments were conducted to evaluate the performance of the proposed adaptive step size, multivariable extremum seeking method, and can be seen in the video: https://youtu.be/xuT-eGATesA. We show that the proposed method is able to find the optimal sideslip and speed for different payloads and can converge more than 30% faster than the standard extremum seeking controller.

A. Experimental setup
A custom-built quadcopter, as shown in Fig. 1 was used during the experiments. The vehicle weighs 660 grams without payload. The distance between the hubs of two diagonal motors is 330 mm and the propeller is 203 mm in diameter. We used two payloads during the experiment, one is a 255×180×85 mm cardboard box weighing 120 grams; the other one is an American football weighing 329 grams. A Crazyflie 2.0 running a custom version of PX4 firmware was used as the low-level flight controller for the vehicle. The experiments were conducted in an indoor flight space with a size of 7×6×5 m. A motion capture system was used for the state estimation of the vehicle and a voltage and current measurement module was connected to the battery to measure the instantaneous power consumption of the vehicle.
The value of parameters of the standard extremum seeking controller and the proposed adaptive step size extremum seeking controller used throughout the experiments are shown in Table I. The perturbation frequency of reference speed w v was set to 1 rad/s, and the perturbation frequency of reference sideslip w s was set to 0.5 rad/s. Multivariable extremum seeking control requires w v = w s [16]. While increasing the perturbation frequencies is helpful to improve the convergence rate of the extremum seeking controller, one should make sure they are not too large for the vehicle to track. The cutoff frequencies of the high-pass and lowpass filters were set to be the same as their corresponding perturbation frequency. The magnitude of speed perturbation a v was set to be 0.15 m/s and the magnitude of sideslip was set to 7.5 • . These values need to be selected large enough to provide the extremum seeking controller with gradient information of the cost function and also make the neighborhood around the optimal value that the extremum seeking controller converges to small. The threshold parameter in Algorithm 1 was empirically set to be 1.
To make a fair comparison, we kept all the control parameters for the two different methods to be the same except k v and k s , since they have different meanings for the two methods: the k v and k s values are the step sizes for the standard method but are only part of the step sizes for the adaptive method, as shown in Section III-B. They were empirically tuned in experiments for the two different methods to achieve the fastest convergence rate while guaranteeing the stability of the system (too large k v and k s will make the system unstable).

B. Performance evaluation
In the experiments, the quadcopter was commanded to fly along a circular path with constant height and a radius of 1.7 meters due to the space constraint. The cost function used is derived in (10), where the power measurement p was provided by the onboard voltage and current sensor and the speed v and was provided by the state estimator based on the motion capture system. To verify that the vehicle converges to the optimal speed and sideslip, we experimentally evaluated the cost function and provided the ground truth values in Fig. 7.
When the cardboard box was used as payload, the performance of the proposed and standard extremum seeking controller is compared in Fig. 5  close to the optimal values shown in the first row of Fig. 7.
When the football was used as payload, the performance of the two methods is compared in Fig. 6. In the first test, the proposed method converged in about 75 s and the standard method converged in about 125 s; in the second test, the proposed method converged in about 100 s and the standard method converge in about 150 s. Both methods converged to speed and sideslip close to the optimal values shown in the second row of Fig. 7.
The experiments have shown that the proposed method can find the optimal range speed and sideslip despite different payloads and initial conditions, and converges more than 30% faster compared to the standard method.

V. CONCLUSION AND FUTURE WORKS
An online, adaptive method for finding the speed and sideslip that maximize the flight range of multicopters is proposed in this work. Not dependent on any power consumption model of the vehicle, it can adapt to disturbances such as different payloads and is easy to deploy. The proposed method can mitigate the common problem of limited flight range of multicopters and thus improve their autonomy. Based on a novel multivariable extremum seeking controller with adaptive step size, it can achieve faster convergence compared to the standard extremum seeking controller with fixed step size. In addition, compared to our previous work of searching for the optimal speed only [18], this work takes finding the optimal sideslip into account and is able to further improve the flight range. Through indoor experiments, we show that this method is able to find the optimal speed and sideslip correctly under different payloads and initial conditions, and it converges more than 30% faster compared to the standard method.
In future work, we plan to do outdoors experiments to further validate the effectiveness of this method, where accurate state estimation from the motion caption system is not available. In addition, the vehicle can fly a circular path with a much larger radius outdoors, which reduces the effect of centripetal force on power consumption.