Titulo Estágio
BCIIDE - a study of brain-machine interfaces for aiding programming tasks.
Áreas de especialidade
Sistemas de Informação
Engenharia de Software
Local do Estágio
DEI-FCTUC
Enquadramento
One of the most exciting current advancements in the human-computer interaction (HCI) research field is the use of non-invasive brain-computer interfaces (BMI) based on electroencephalography (EEG) devices. Current EEG technology allied with the contemporary computer processing power that can perform digital signal processing (DSP) and machine learning (ML) algorithms in real time have fomented the creation of several research experiments focused on better understanding and discriminating the brain’s activity when performing distinct tasks or responding to specific external stimuli. This technology has been most explored and implemented in the field of clinical rehabilitation and assistive HCI, leading to the development of applications that help improve the quality of life of people with severe motor and cognitive impairments [1].
The potential use of EEG based BCI for quotidian use, as a “standard” computer interface that works side-by-side with the mouse and keyboard, is still a topic that needs to be explored and experimented from an HCI perspective. In the specific context of software programming, passive BCI strategies have been used to measure developers’ cognitive load, which in turn is related to software quality and errors. Existing research has been focused on software comprehension tasks, thus, there is a lack of research and development of interactive devices using EEG based BCI for enhancing programming performances.
Objetivo
This project’s main objective is to explore the use of EEG based BCI as an interaction device for enhancing productivity in quotidian programming tasks. The goal of the master thesis research is to assess the current state-of-art in EEG signal processing and user centered HCI design and answer the following research question:
Can BCI systems improve or enhance user interaction and productivity in software programming tasks?
To answer the proposed questions, we will implement an experimental design of the HCI domain, where we will quantitatively evaluate users’ and system performance and qualitatively evaluate user experience while performing software tasks in a custom built IDE, presenting 4 independent interaction possibilities: standard mouse+keyboard interaction; passive BCI capable of monitoring the user cognitive and emotional state and predicting when the user is “stuck” and needs assistance with the programming problem; active BCI capable of detecting spontaneous predetermined commands such as “save document” or “build/run code”, associated to pre-trained (user and algorithm) mental imagery; reactive BCI capable of detecting users’ unconscious brain activity as a response to specific visual stimuli (SSVEP strategy) for selecting between different menu items that are presented on the screen.
From the main research question and methodology, we derive the following practical objectives:
1-Develop a minimalist IDE that allows the user to write and compile Python code and allows the recording of timestamped user input actions (keystroke, mouse click, IDE actions) that can be synchronized to EEG data.
2-Develop a set of experiments for evaluating the IDE and user performance in programming tasks.
Plano de Trabalhos - Semestre 1
1- Study the BMI and evaluation of the relations between programming errors and programmers cognitive load.
2- Develop the experimental IDE.
3- Get familiarized with EEG equipment.
4- Submit experimental protocol documentation to ethical committee.
Plano de Trabalhos - Semestre 2
1- Implement BMI features in the IDE.
2- Implement experimental trials with subjects (DEI students).
3- Analyze results.
4- write/submit conference paper
Condições
The work will be supervise by prof. Dr. André Perrotta(supervisor) and Dra. Fuqun Huang(co-supervisor)
A proportion of the proposed work may be funded by a three-month or six-month grant (subject to application and acceptance at open call for MS grant).
The research will have access to a research/medical grade EEG system (BrainVision V-amp 16 channels with electrodes and accessories).
The supervising team have rich experience in the areas of HCI, BMI, Programming Psychology, and Human Errors.
Pais-Vieira, Carla, Pedro Gaspar, Demétrio Matos, Leonor Palminha Alves, Bárbara Moreira da Cruz, Maria João Azevedo, Miguel Gago, Tânia Poleri, André Perrotta, και Miguel Pais-Vieira. ‘Embodiment Comfort Levels During Motor Imagery Training Combined With Immersive Virtual Reality in a Spinal Cord Injury Patient’. Frontiers in Human Neuroscience 16 (2022). https://doi.org/10.3389/fnhum.2022.909112.
Perrotta, A., Pais-Vieira, C., Allahdad, M. K., Bicho, E., & Pais-Vieira, M. (2020). Differential width discrimination task for active and passive tactile discrimination in humans. MethodsX, 7, 100852.
Huang, F., & Strigini, L. (2021). HEDP: A Method for Early Forecasting Software Defects based on Human Error Mechanisms. arXiv preprint arXiv:2110.06758. (to appear in IEEE Access in 2022)
Huang, Fuqun, and Henrique Madeira. "Targeted Code Inspection based on Human Errors." In 2021 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), pp. 274-275. IEEE, 2021.
Huang, F., Liu, B., Song, Y., & Keyal, S. (2014). The links between human error diversity and software diversity: Implications for fault diversity seeking. Science of Computer Programming, 89, 350-373.
Observações
The preferred candidate will have the following characteristics:
*-Proficient in reading and writing in English.
*-Good programming skills.
*-Working knowledge of digital signal processing
*-Interested in BCI
Bibliography
[1] M. A. Lebedev e M. A. L. Nicolelis, “Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation,” Physiological Reviews, vol. 97, pp. 767-837, 4 2017.
[2] M. V. Kosti, K. Georgiadis, D. A. Adamos, N. Laskaris, D. Spinellis e L. Angelis, “Towards an affordable brain computer interface for the assessment of programmers' mental workload,” International Journal of Human-Computer Studies, vol. 115, pp. 52-66, 7 2018.
[3] J. Medeiros, R. Couceiro, G. Duarte, J. Durães, J. Castelhano, C. Duarte, M. Castelo-Branco, H. Madeira, P. Carvalho e C. Teixeira, “Can EEG Be Adopted as a Neuroscience Reference for Assessing Software Programmers' Cognitive Load?,” Sensors, vol. 21, p. 2338, 3 2021.
[4] S. C. Muller e T. Fritz, “Stuck and Frustrated or in Flow and Happy: Sensing Developers Emotions and Progress,” em 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, 2015.
Orientador
André Venturoti Perrotta
avperrotta@dei.uc.pt 📩