CNNATT: Deep EEG & fNIRS Real-Time Decoding of bimanual forces
  Non-invasive cortical neural interfaces have only achieved modest performance
in cortical decoding of limb movements and their forces, compared to invasive
brain-computer interfaces (BCIs). While non-invasive methodologies are safer,
cheaper and vastly more accessible technologies, signals suffer from either
poor resolution in the space domain (EEG) or the temporal domain (BOLD signal
of functional Near Infrared Spectroscopy, fNIRS). The non-invasive BCI decoding
of bimanual force generation and the continuous force signal has not been
realised before and so we introduce an isometric grip force tracking task to
evaluate the decoding. We find that combining EEG and fNIRS using deep neural
networks works better than linear models to decode continuous grip force
modulations produced by the left and the right hand. Our multi-modal deep
learning decoder achieves 55.2 FVAF[%] in force reconstruction and improves the
decoding performance by at least 15% over each individual modality. Our results
show a way to achieve continuous hand force decoding using cortical signals
obtained with non-invasive mobile brain imaging has immediate impact for
rehabilitation, restoration and consumer applications.
