A novel method for extracting interpretable knowledge from a spiking
  neural classifier with time-varying synaptic weights
  This paper presents a novel method for information interpretability in an
MC-SEFRON classifier. To develop a method to extract knowledge stored in a
trained classifier, first, the binary-class SEFRON classifier developed earlier
is extended to handle multi-class problems. MC-SEFRON uses the population
encoding scheme to encode the real-valued input data into spike patterns.
MC-SEFRON is trained using the same supervised learning rule used in the
SEFRON. After training, the proposed method extracts the knowledge for a given
class stored in the classifier by mapping the weighted postsynaptic potential
in the time domain to the feature domain as Feature Strength Functions (FSFs).
A set of FSFs corresponding to each output class represents the extracted
knowledge from the classifier. This knowledge encoding method is derived to
maintain consistency between the classification in the time domain and the
feature domain. The correctness of the FSF is quantitatively measured by using
FSF directly for classification tasks. For a given input, each FSF is sampled
at the input value to obtain the corresponding feature strength value (FSV).
Then the aggregated FSVs obtained for each class are used to determine the
output class labels during classification. FSVs are also used to interpret the
predictions during the classification task. Using ten UCI datasets and the
MNIST dataset, the knowledge extraction method, interpretation and the
reliability of the FSF are demonstrated. Based on the studies, it can be seen
that on an average, the difference in the classification accuracies using the
FSF directly and those obtained by MC-SEFRON is only around 0.9% & 0.1\% for
the UCI datasets and the MNIST dataset respectively. This clearly shows that
the knowledge represented by the FSFs has acceptable reliability and the
interpretability of classification using the classifier's knowledge has been
justified.
