Titulo Estágio
Next-Generation Lyrics Music Emotion Variation Detection
Local do Estágio
DEI-FCTUC
Enquadramento
The area of Music Emotion Recognition (MER) is part of a broader area called Music Information Retrieval (MIR) and is increasingly in vogue in the scientific community and industry. It has immense applications in medicine (e.g., treatment of certain pathologies), entertainment (e.g., cinema), advertising, music platforms (e.g., Spotify, Apple Music), etc.
Music is made up of two main dimensions (audio and lyrics), and both usually play an important role in the emotional perception that a listener has of a song, always ultimately depending on the listener and the song.
There are two main problems normally associated with the MER process: Classifying a song (audio + lyrics) according to its predominant emotion and Music Emotion Variation Detection (audio + lyrics) (MEVD). This last problem has several practical applications (in addition to those listed above) that include all those in which it is important to understand the emotion transmitted throughout the song (e.g., transmitting to the person singing karaoke the emotion that each part of the music expresses).
There are several state-of-the-art approaches to this MER problem, the most typical of which is separating the problem into two dimensions (audio and lyrics). In the case of song lyrics (Lyrics Music Emotion Variation Detection—LMEVD), the objective is to identify the variation in emotion throughout the verses of the song lyrics.
The objective of this project is to build on work previously carried out in the MIR-CISUC group to continue the work and propose new approaches and methodologies to attack the problem of Lyrics Music Emotion Variation Detection (LMEVD), which, in addition to the previous process, can include the creation of new features for the song lyrics. The project may also include participation in the creation of a corpus of songs prepared for MEVD. The work to be developed involves the use of natural language processing techniques, text mining, feature engineering, and knowledge of applied machine learning.
Objetivo
The objectives to be achieved in this project are the following:
1. Research and development of new emotionally based features for Lyrics Music Emotion Variation Detection (LMEVD).
2. Research and development of NLP approaches associated with LMEVD.
3. Participation in the creation of a corpus of songs prepared for LMEVD.
Plano de Trabalhos - Semestre 1
1. State-of-the-art review: MER, NLP and feature engineering, machine learning for LMEVD
2. Participation in the creation of a corpus for LMEVD
3. Research and development of NLP and feature engineering approaches for LMEVD
4. Writing of the Intermediate Report
Plano de Trabalhos - Semestre 2
5. Research and development of deep learning (DL) approaches for LMEVD (e.g., via CNNs, transfer learning, etc.)
6. Critical evaluation of the developed DL models
7. Writing of a scientific paper and thesis
Condições
- Access to a toolbox for Lyrics Music Emotion Recognition (with a feature extraction module), developed by our team
- Access to an LMEVD database, developed by our team
- Server access (hosted at DEI) with 10 high-performance GPU cards
Orientador
Rui Pedro Paiva, Ricardo Malheiro e Renato Panda
ruipedro@dei.uc.pt 📩