Titulo Estágio
Feature Engineering and Learning for Audio-based Music Emotion Variation Detection
Local do Estágio
DEI-FCTUC
Enquadramento
Given its major social importance, particularly in the digital society, music plays an important role in the world economy. In 2017, digital music revenues rose to US$7.8 billion, accounting for 50% of the global music revenues, while the global recorded music market grew by 5.9%.
It is expected that the amount of digital music continues to explode. Digital music repositories need, then, more advanced, flexible and user-friendly search mechanisms, adapted to the requirements of individual users. This has led to an increased awareness to the Music Information Retrieval (MIR) area. Several companies, e.g., Google, Pandora, Spotify, Apple, Sony or Philips, have set up MIR research agendas, with commercial applications already in place.
Within MIR, Music Emotion Recognition (MER) emerged as a significant sub-field. In fact, “music’s preeminent functions are social and psychological”, and so “the most useful retrieval indexes are those that facilitate searching in conformity with such social and psychological functions. Typically, such indexes will focus on stylistic, mood, and similarity information”.
Besides the music industry, the range of applications of MER is wide and varied, e.g., game development, cinema, advertising or health (e.g., music therapy).
MER research promises significant social, economic and cultural repercussions, as well as substantial scientific impact. There are presently several open and complex research problems in this multidisciplinary field, touching fields such as audio signal processing, natural language processing (NLP), feature engineering and machine learning.
Current MER solutions, either in the audio or lyrics domain, are still unable to accurately solve fundamental problems, such as classification of static samples (i.e., samples with uniform emotion) with a single label, in few emotion classes (e.g., four to five). This is supported by both existing studies and the small improvements in the Music Information Retrieval Evaluation eXchange (MIREX) Audio Mood Classification (AMC) task (www.music-ir.org/mirex/), where results stabilized at around 69% accuracy for several years.
In this context, both classical feature engineering and deep learning approaches appear as a promising for the study of Music Emotion Variation Detection (MEVD), i.e., the analysis of emotion variations throughout a complete song.
Objetivo
The objectives to be achieved in this project are the following:
1. Research and development of audio analysis and feature engineering for MEVD
2. Research and development of DL approaches for MEVD
Plano de Trabalhos - Semestre 1
1. State-of-the-art review: MER and MEVD, deep learning for audio in general and MEVD in particular
2. Update of the current MEVD dataset (previously created by our group)
3. Evaluation of the current algorithms (based on classical feature engineering and machine learning) on the novel datasets
4. Research and development of audio analysis and feature engineering for MEVD (e.g., emotion-based music segmentation, structural song analysis, musical form features)
5. Writing of the Intermediate Report
Plano de Trabalhos - Semestre 2
6. Continuation of dataset updating
7. Continuation of research and development of audio analysis and feature engineering for MEVD
8. Research and development of DL approaches for MEVD (e.g., via LSTMs, etc.)
9. Critical evaluation of the developed DL models
10. Writing of a scientific paper and thesis
Condições
Possibility of scholarship in the second semester (6 months, 930.98 EUR/month). At this moment, this is a likely possibility, but that still needs to be confirmed.
Orientador
Rui Pedro Paiva, Renato Panda e Ricardo Malheiro
ruipedro@dei.uc.pt 📩