Titulo Estágio
Bimodal Automatic Song Structure Segmentation and Classification for Music Emotion Recognition
Áreas de especialidade
Sistemas Inteligentes
Sistemas Inteligentes
Local do Estágio
DEI/FCTUC
Enquadramento
The area of Music Emotion Recognition (MER), a sub-area of Music Information Retrieval (MIR), has received increasing interest from the international scientific community, both academic and industrial. In fact, companies such as Spotify, Pandora, Google, Sony, Philips Research, and Gracenote, among many others, have invested heavily in the area in recent years and used this aspect in their applications.
The recognition of music in emotions contemplates the recognition of emotions in the two dimensions of music (audio and song lyrics). This is because the emotional perception that a listener has of a song can depend on any of these two dimensions with variable weights, always depending on ultimately the listener and the music.
The classic process of building prediction models for MER begins by extracting features from the audio and lyrics separately. For example, in audio, we have features related to the detection of rhythm, melody, tempo, etc., and for lyric features, we have semantic,
Objetivo
The objectives to be achieved in this project are the following:
1. Research and development of musical structures commonly associated with music.
2. Research and development of automatic methods for song structural segmentation and classification
Plano de Trabalhos - Semestre 1
1. State-of-the-art review: MER, Song structure analysis, song structure segmentation and classification
2. Evaluation and implementation of current algorithms for song structure segmentation and classification based on audio (classical and DL-based)
3. Writing of the Intermediate Report
Plano de Trabalhos - Semestre 2
4. Evaluation and implementation of current algorithms for song structure segmentation and classification based on lyrics (classical and DL-based)
5. Proposal of a novel bimodal algorithm for ong structure segmentation and classification based on lyrics (classical and/or DL-based)
6. Critical evaluation of the developed models
7. Writing of a scientific paper and thesis
Condições
- Access to a toolbox for audio and lyrics MER (with feature extraction and deep learning modules), developed by our team
- Access to a bimodal MER database, developed by our team
- Server access (hosted at DEI) with 10 high-performance GPU cards
Observações
n/a
Orientador
Rui Pedro Paiva, Ricardo Malheiro e Renato Panda
ruipedro@dei.uc.pt 📩