Titulo Estágio
MERGE Lyrics: Music Emotion Recognition next Generation – Lyrics Classification with Deep Learning
Áreas de especialidade
Sistemas Inteligentes
Local do Estágio
DEI
Enquadramento
Given its major social importance, particularly in the digital society, music plays an important role in the world economy. In 2017, digital music revenues rose to US$7.8 billion, accounting for 50% of the global music revenues, while the global recorded music market grew by 5.9%.
It is expected that the amount of digital music continues to explode. Digital music repositories need, then, more advanced, flexible and user-friendly search mechanisms, adapted to the requirements of individual users. This has led to an increased awareness to the Music Information Retrieval (MIR) area. Several companies, e.g., Google, Pandora, Spotify, Apple, Sony or Philips, have set up MIR research agendas, with commercial applications already in place.
Within MIR, Music Emotion Recognition (MER) emerged as a significant sub-field. In fact, “music’s preeminent functions are social and psychological”, and so “the most useful retrieval indexes are those that facilitate searching in conformity with such social and psychological functions. Typically, such indexes will focus on stylistic, mood, and similarity information”.
Besides the music industry, the range of applications of MER is wide and varied, e.g., game development, cinema, advertising or health (e.g., music therapy).
MER research promises significant social, economic and cultural repercussions, as well as substantial scientific impact. There are presently several open and complex research problems in this multidisciplinary field, touching fields such as audio signal processing, natural language processing (NLP), feature engineering and machine learning.
Current MER solutions, either in the audio or lyrics domain, are still unable to accurately solve fundamental problems, such as classification of static samples (i.e., samples with uniform emotion) with a single label, in few emotion classes (e.g., four to five). This is supported by both existing studies and the small improvements in the Music Information Retrieval Evaluation eXchange (MIREX) Audio Mood Classification (AMC) task (www.music-ir.org/mirex/), where results stabilized at around 69% accuracy for several years.
In this context, deep learning appears as a promising approach, particularly for the study of Lyrics Music Emotion Variation Detection (LMEVD), i.e., the analysis of emotion variations throughout a complete song.
Objetivo
The objectives to be achieved in this project are the following:
1. Research and development of deep learning (DL) approaches for static lyrics-based MER (LMER)
2. Research and development of DL approaches for LMEVD
Plano de Trabalhos - Semestre 1
1. State-of-the-art review: MER, deep learning for text in general and LMER in particular
2. Update of the current LMER and LMEVD datasets (previously created by our group)
3. Evaluation of the current algorithms (based on classical Natural Language Processing techniques and machine learning) on the novel datasets
4. Writing of the Intermediate Report
Plano de Trabalhos - Semestre 2
5. Continuation of dataset updating
6. Research and development of deep learning (DL) approaches for LMER (e.g., via CNNs, transfer learning, etc.)
7. Research and development of DL approaches for LMEVD (e.g., via LSTMs, etc.)
8. Critical evaluation of the developed DL models
9. Writing of a scientific paper and thesis
Condições
Possibility of scholarship in the second semester (6 months, 835.98 EUR/month). At this moment, this is a likely possibility, but that still needs to be confirmed.
Orientador
Rui Pedro Paiva, Prof. Ricardo Malheiro
ruipedro@dei.uc.pt 📩