Forward Composition Propagation for Explainable Neural Reasoning
  This paper proposes an algorithm called Forward Composition Propagation (FCP)
to explain the predictions of feed-forward neural networks operating on
structured pattern recognition problems. In the proposed FCP algorithm, each
neuron is described by a composition vector indicating the role of each problem
feature in that neuron. Composition vectors are initialized using a given input
instance and subsequently propagated through the whole network until we reach
the output layer. It is worth mentioning that the algorithm is executed once
the network's training network is done. The sign of each composition value
indicates whether the corresponding feature excites or inhibits the neuron,
while the absolute value quantifies such an impact. Aiming to validate the FCP
algorithm's correctness, we develop a case study concerning bias detection in a
state-of-the-art problem in which the ground truth is known. The simulation
results show that the composition values closely align with the expected
behavior of protected features.
