Learnability and Robustness of Shallow Neural Networks Learned With a
  Performance-Driven BP and a Variant PSO For Edge Decision-Making
  In many cases, the computing resources are limited without the benefit from
GPU, especially in the edge devices of IoT enabled systems. It may not be easy
to implement complex AI models in edge devices. The Universal Approximation
Theorem states that a shallow neural network (SNN) can represent any nonlinear
function. However, how fat is an SNN enough to solve a nonlinear
decision-making problem in edge devices? In this paper, we focus on the
learnability and robustness of SNNs, obtained by a greedy tight force heuristic
algorithm (performance driven BP) and a loose force meta-heuristic algorithm (a
variant of PSO). Two groups of experiments are conducted to examine the
learnability and the robustness of SNNs with Sigmoid activation,
learned/optimised by KPI-PDBPs and KPI-VPSOs, where, KPIs (key performance
indicators: error (ERR), accuracy (ACC) and $F_1$ score) are the objectives,
driving the searching process. An incremental approach is applied to examine
the impact of hidden neuron numbers on the performance of SNNs,
learned/optimised by KPI-PDBPs and KPI-VPSOs. From the engineering prospective,
all sensors are well justified for a specific task. Hence, all sensor readings
should be strongly correlated to the target. Therefore, the structure of an SNN
should depend on the dimensions of a problem space. The experimental results
show that the number of hidden neurons up to the dimension number of a problem
space is enough; the learnability of SNNs, produced by KPI-PDBP, is better than
that of SNNs, optimized by KPI-VPSO, regarding the performance and learning
time on the training data sets; the robustness of SNNs learned by KPI-PDBPs and
KPI-VPSOs depends on the data sets; and comparing with other classic machine
learning models, ACC-PDBPs win for almost all tested data sets.
