Titulo Estágio
Disguised source animations in steady-state-visually-evoked- potential based BCI games
Áreas de especialidade
Sistemas de Informação
Sistemas Inteligentes
Local do Estágio
DEI
Enquadramento
Brain computer interfaces (BCI) based games is a “hot” topic in the context of neurorehabilitation research, especially in the cases where the interface uses electroencephalogram (EEG) signals as the main data source to be processed for detecting the user’s actions and commands. In this context, one of the most reliable and utilized strategies is to track the distinguished EEG signal patterns that are generated on the user’s brain occipital region as a result to an animated visual stimulus of specific frequencies. This technique is thus called steady-state-visually-evoked-potential (SSVEP) and it has been widely used due to its reliability, ease of signal processing and pre-processing requirements and it is effective for any healthy user of any age and genre.
The most common stimulus used in SSVEP BCI is a flickering animation of a checkerboard that “blinks” from one state to another in a pre-defined frequency. The “blinking” frequency provokes a proportional response on the EEG sensors placed above the occipital region (back of the head) that can be tracked by analyzing the peaks of the frequency spectrum of the EEG signal in these specific channels. For this strategy to work, several parameters and conditions must be met, such as the size of the animation pattern in the visual field of the user, how focused the user is on the pattern and the amount of time the user focuses the attention on the pattern. Consequently, most SSVEP based BCI applications require that some part of the screen be covered with the stimulus patterns and animations, which in applications that present a strong interactive and aesthetically creative aspect, such as games, might hinder some important aspects such as enjoyment, user engagement and visual comfort. To address this issue, attempts in generating different types of animations have been tried, such as rotating patterns, rotating images, flickering entire objects/elements of the screen and grow/shrink images. Nevertheless, these approaches still result in the use of aesthetically “odd” visual elements that are usually incongruent with the background and other visual elements on the screen.
In this matter, a strategy to generate animated visual patterns that meet the technical parameters and at the same time are disguised as animations and effects that seamlessly integrate the whole visual scenery and other elements of the application would be an interesting innovation, especially when focused on the development of SSVEP based BCI games. To explore this idea, two strategies could be experimented: procedurally generated scene elements such as trees, water features, clouds, etc., animated with the desired frequencies and strategically positioned and blended with other static elements; visual effects applied to the ordinary scene elements, such as warping, rippling, rotating colors.
Objetivo
• Study and understand the state-of-art of EEG based BCI, with special focus on games and serious games
• Develop animations and animated effects for experimenting SSVEP with “disguised” elements
• Develop 2 different scenarios for experimenting with “disguised” SSVEP elements: one urban navigation, one nature navigation
Plano de Trabalhos - Semestre 1
1. Study the state-of-art of existing SSVEP based BCI games
2. Develop procedurally generated elements for testing SSVEP performance:
a. Nature context: trees, bushes, grass, clouds, water features
b. Urban Context: animated elements in building facades (windows, doors, graffiti)
3. Develop visual animated effects for modifying ordinary elements: warping, rippling, changing colors
4. Write intermediate report
Plano de Trabalhos - Semestre 2
5. Experiment with subjects to test the performance of the generated elements and compare it to a control group.
6. Develop 2 simple VR navigation/game that use the best performance elements
7. Test the VR applications with subjects and evaluate performance and visual comfort
8. Write final report
Condições
A equipa terá acesso a utilização de um equipamento EEG de nível profissional e portátil: amplificador brainVision V-amp, set de 16 eletrodos, toucas de diversos tamanhos, gel, sistema de calibração, software de aquisição.
O orientador tem experiência na utilização deste sistema e participa em projetos multidisciplinares utilizando BCI.
Candidate’s profile:
The preferred candidate will have the following characteristics:
• Proficient in reading and writing in English
• Good programming and creative coding skills
• Working knowledge of digital signal processing
• Interested in developing VR and interactive applications
• Interested in BCI
Observações
Co-orientador: Jorge C. S. Cardoso jorgecardoso@dei.uc.pt
Bibliography
Perrotta, A., Pais-Vieira, C., Allahdad, M. K., Bicho, E., & Pais-Vieira, M. (2020). Differential width discrimination task for active and passive tactile discrimination in humans. MethodsX, 7, 100852.
Coordenador de tecnologia no projeto Thertact-exo, Prêmio Santa Casa Neurociências 2018
Marshall, D., Coyle, D., Wilson, S., & Callaghan, M. (2013). Games, gameplay, and BCI: the state of the art. IEEE Transactions on Computational Intelligence and AI in Games, 5(2), 82-99.
Rekrut, M., Jungbluth, T., Alexandersson, J., & Krüger, A. (2021, April). Spinning Icons: Introducing a Novel SSVEP-BCI Paradigm Based on Rotation. In 26th International Conference on Intelligent User Interfaces (pp. 234-243).
Choi, K. M., Park, S., & Im, C. H. (2019). Comparison of visual stimuli for steady-state visual evoked potential-based brain-computer interfaces in virtual reality environment in terms of classification accuracy and visual comfort. Computational intelligence and neuroscience, 2019.
Kübler, A., Holz, E. M., Riccio, A., Zickler, C., Kaufmann, T., Kleih, S. C., ... & Mattia, D. (2014). The user-centered design as novel perspective for evaluating the usability of BCI-controlled applications. PloS one, 9(12), e112392.
Zhu, D., Bieger, J., Garcia Molina, G., & Aarts, R. M. (2010). A survey of stimulation methods used in SSVEP-based BCIs. Computational intelligence and neuroscience, 2010.
Orientador
André Perrotta
avperrotta@dei.uc.pt 📩