Reinforcement Learning-Based Joint Self-Optimisation Method for the
  Fuzzy Logic Handover Algorithm in 5G HetNets
  5G heterogeneous networks (HetNets) can provide higher network coverage and
system capacity to the user by deploying massive small base stations (BSs)
within the 4G macro system. However, the large-scale deployment of small BSs
significantly increases the complexity and workload of network maintenance and
optimisation. The current handover (HO) triggering mechanism A3 event was
designed only for mobility management in the macro system. Directly
implementing A3 in 5G-HetNets may degrade the user mobility robustness.
Motivated by the concept of self-organisation networks (SON), this study
developed a self-optimised triggering mechanism to enable automated network
maintenance and enhance user mobility robustness in 5G-HetNets. The proposed
method integrates the advantages of subtractive clustering and Q-learning
frameworks into the conventional fuzzy logic-based HO algorithm (FLHA).
Subtractive clustering is first adopted to generate a membership function (MF)
for the FLHA to enable FLHA with the self-configuration feature. Subsequently,
Q-learning is utilised to learn the optimal HO policy from the environment as
fuzzy rules that empower the FLHA with a self-optimisation function. The FLHA
with SON functionality also overcomes the limitations of the conventional FLHA
that must rely heavily on professional experience to design. The simulation
results show that the proposed self-optimised FLHA can effectively generate MF
and fuzzy rules for the FLHA. By comparing with conventional triggering
mechanisms, the proposed approach can decrease the HO, ping-pong HO, and HO
failure ratios by approximately 91%, 49%, and 97.5% while improving network
throughput and latency by 8% and 35%, respectively.
