Towards EEG/fNIRS-based semantic brain-computer interfacing

Semantic neural decoding aims to identify which semantic concepts an individual is focused on at a given moment in time from recordings of their brain activity. This could be used by brain-computer interfaces (BCIs) for communication. These semantic BCIs have the potential to be highly intuitive by allowing direct communication of semantic concepts instead of spelling one character at a time, as is the case with current state-of-the-art BCI systems. This thesis explores the feasibility of semantic BCIs based on electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). We designed an experiment to differentiate between the semantic categories of animals and tools during a silent naming task (for the first time in fNIRS), and three novel and intuitive sensory-based imagery tasks using visual, auditory, and tactile perception. Participants were asked to visualize an object in their minds, imagine the sounds made by the object, and imagine the feeling of touching the object. We showed the possibility of semantic neural decoding in both neuroimaging modalities but with contrasting differences in comparison with other state-of-the-art research. Furthermore, we investigated the influence of cue presentation on EEG-based semantic decoding. We found that all EEG-based semantic decoding studies published to date could exploit neural activity recorded during the cue presentation period in their analyses. We showed that including the cue presentation period in the classification pipeline significantly increases classification accuracies. While this research area involves considerable challenges, this thesis made a step towards EEG/fNIRS-based semantic BCIs.