Titulo Estágio
MERGE Audio: Audio Analysis and Feature Engineering for Music Emotion Recognition
Áreas de especialidade
Sistemas Inteligentes
Local do Estágio
DEI-FCTUC
Enquadramento
Given its major social importance, particularly in the digital society, music plays an important role in the world economy. In 2017, digital music revenues rose to US$7.8 billion, accounting for 50% of the global music revenues, while the global recorded music market grew by 5.9%.
It is expected that the amount of digital music continues to explode. Digital music repositories need, then, more advanced, flexible and user-friendly search mechanisms, adapted to the requirements of individual users. This has led to an increased awareness to the Music Information Retrieval (MIR) area. Several companies, e.g., Google, Pandora, Spotify, Apple, Sony or Philips, have set up MIR research agendas, with commercial applications already in place.
Within MIR, Music Emotion Recognition (MER) emerged as a significant sub-field. In fact, “music’s preeminent functions are social and psychological”, and so “the most useful retrieval indexes are those that facilitate searching in conformity with such social and psychological functions. Typically, such indexes will focus on stylistic, mood, and similarity information”.
Besides the music industry, the range of applications of MER is wide and varied, e.g., game development, cinema, advertising or health (e.g., music therapy).
MER research promises significant social, economic and cultural repercussions, as well as substantial scientific impact. There are presently several open and complex research problems in this multidisciplinary field, touching fields such as audio signal processing, natural language processing (NLP), feature engineering and machine learning.
Current MER solutions, either in the audio or lyrics domain, are still unable to accurately solve fundamental problems, such as classification of static samples (i.e., samples with uniform emotion) with a single label, in few emotion classes (e.g., four to five). This is supported by both existing studies and the small improvements in the Music Information Retrieval Evaluation eXchange (MIREX) Audio Mood Classification (AMC) task (www.music-ir.org/mirex/), where results stabilized at around 69% accuracy for several years.
In this context, deep learning appears as a promising approach, particularly for the study of Music Emotion Variation Detection (MEVD), i.e., the analysis of emotion variations throughout a complete song.
Objetivo
The objectives to be achieved in this project are the following:
1. Research and development of audio analysis and feature engineering approaches for static MER; deep learning (DL) approaches are a secondary goal to be addresses, if time so permits
2. Research and development of audio analysis and feature engineering for MEVD; again, deep learning (DL) approaches are a secondary goal to be addressed, if time so permits
3. Update of the current MERGE app prototype
Plano de Trabalhos - Semestre 1
1. State-of-the-art review: MER, audio analysis and feature engineering, deep learning for audio in general and MER in particular
2. Update of the current MER and MEVD datasets (previously created by our group)
3. Evaluation of the current algorithms (based on classical feature engineering and machine learning) on the novel datasets
4. Requirement and architectural analysis of the improved MERGE app
5. Writing of the Intermediate Report
Plano de Trabalhos - Semestre 2
6. Continuation of dataset updating
7. Research and development of audio analysis and feature engineering (and, if possible, DL) approaches for static MER (namely, handractfing of expressivity and musical texture features)
8. Research and development of audio analysis and feature engineering (and, if possible, DL) for MEVD (namely, handcrafting of musicl form features)
9. Critical evaluation of the developed feature engineering models
10. Writing of a scientific paper and thesis
Condições
Possibility of scholarship in the second semester (6 months, 875.98 EUR/month), within the MERGE project.
Orientador
Rui Pedro Paiva, Renato Panda e Ricardo Malheiro
ruipedro@dei.uc.pt 📩