Fully-echoed Q-routing with Simulated Annealing Inference for Flying
  Adhoc Networks
  Current networking protocols deem inefficient in accommodating the two key
challenges of Unmanned Aerial Vehicle (UAV) networks, namely the network
connectivity loss and energy limitations. One approach to solve these issues is
using learning-based routing protocols to make close-to-optimal local decisions
by the network nodes, and Q-routing is a bold example of such protocols.
However, the performance of the current implementations of Q-routing algorithms
is not yet satisfactory, mainly due to the lack of adaptability to continued
topology changes. In this paper, we propose a full-echo Q-routing algorithm
with a self-adaptive learning rate that utilizes Simulated Annealing (SA)
optimization to control the exploration rate of the algorithm through the
temperature decline rate, which in turn is regulated by the experienced
variation rate of the Q-values. Our results show that our method adapts to the
network dynamicity without the need for manual re-initialization at transition
points (abrupt network topology changes). Our method exhibits a reduction in
the energy consumption ranging from 7% up to 82%, as well as a 2.6 fold gain in
successful packet delivery rate}, compared to the state of the art Q-routing
protocols
