Titulo Estágio
MOODoke: Emotion-based Karaoke
Áreas de especialidade
Sistemas Inteligentes
Engenharia de Software
Local do Estágio
DEI
Enquadramento
The area of Music Emotion Recognition (MER), a sub-area of the Music Information Retrieval (MIR), is attracting increasing interest by the international scientific community, both academic and industrial. In fact, companies such as Spotify, Pandora, Google, Sony, Philips Research, Gracenote, among others, have invested strongly in the area in recent years.
In particular, MER enables a broad set of applications in fields such as automatic music classification, emotion-based playlist generation and similarity analysis, music therapy, as well as application in the gaming, film and advertising industries, among others.
In this context, as part of a cooperation between the MIR group at the Center of Informatics and Systems of the University of Coimbra (CISUC) and the Laboratory of Music and Computational Audio, Academia Sinica, Taiwan, we developed a computational prototype entitled MOODoke: Emotion-based Karaoke, which can be found at http://moodoke.dei.uc.pt/. The prototype is based on music emotion recognition in the context of karaoke, resorting to YouTube videos. From these videos, song lyrics are automatically extracted through optical character recognition (OCR – this module is already developed), from which lyrics-based music emotion variation detection (MEVD) is to be performed.
Objetivo
So far, only the application front-end and the OCR module have been developed. As such, the objectives to be achieved in this project are the following:
1. To update the current system via:
1.1. The integration of the emotion-based song verses classification module (already developed, but still to integrate)
1.2. The addition of novel features to the module mentioned in a), based on Natural Language Processing (NLP)
1.3. Code cleaning and optimization, to make it publicly available
2. To extend an already available emotion-based song verses dataset, to use in the validation of system’s classification accuracy
3. To evaluate the complete solution, both in terms of software quality requirements (e.g., computational performance) and classification accuracy
4. To write a scientific paper and present it at an international conference
Plano de Trabalhos - Semestre 1
1. Familiarization with the application and code (1 month; Sep/2019)
2. State of the art review (1 month; Oct/2019)
3. Integration of the current emotion-based song verses classification module, code cleaning and optimization (2 months; Nov-Dec/2019)
4. Writing of the intermediate report (1 month; Dec 2019)
Plano de Trabalhos - Semestre 2
4. Dataset acquisition (6 months; Oct/2019 – Mar/2020)
5. Development of novel NLP features for lyrics-based MER (4 months; Jan-Apr/2020)
6. Complete system evaluation (1 month; May/2020)
7. Writing of a scientific paper and thesis (1 month; June/2020)
Condições
Nothing to add.
Orientador
Rui Pedro Paiva, Ricardo Malheiro
ruipedro@dei.uc.pt 📩