Titulo Estágio
MERGE Audio: Music Emotion Recognition next Generation – Audio Classification with Deep Learning
Áreas de especialidade
Sistemas Inteligentes
Local do Estágio
DEI-FCTUC
Enquadramento
Given its major social importance, particularly in the digital society, music plays an important role in the world economy. In 2017, digital music revenues rose to US$7.8 billion, accounting for 50% of the global music revenues, while the global recorded music market grew by 5.9%.
It is expected that the amount of digital music continues to explode. Digital music repositories need, then, more advanced, flexible and user-friendly search mechanisms, adapted to the requirements of individual users. This has led to an increased awareness to the Music Information Retrieval (MIR) area. Several companies, e.g., Google, Pandora, Spotify, Apple, Sony or Philips, have set up MIR research agendas, with commercial applications already in place.
Within MIR, Music Emotion Recognition (MER) emerged as a significant sub-field. In fact, “music’s preeminent functions are social and psychological”, and so “the most useful retrieval indexes are those that facilitate searching in conformity with such social and psychological functions. Typically, such indexes will focus on stylistic, mood, and similarity information”.
Besides the music industry, the range of applications of MER is wide and varied, e.g., game development, cinema, advertising or health (e.g., music therapy).
MER research promises significant social, economic and cultural repercussions, as well as substantial scientific impact. There are presently several open and complex research problems in this multidisciplinary field, touching fields such as audio signal processing, natural language processing (NLP), feature engineering and machine learning.
Current MER solutions, either in the audio or lyrics domain, are still unable to accurately solve fundamental problems, such as classification of static samples (i.e., samples with uniform emotion) with a single label, in few emotion classes (e.g., four to five). This is supported by both existing studies and the small improvements in the Music Information Retrieval Evaluation eXchange (MIREX) Audio Mood Classification (AMC) task (www.music-ir.org/mirex/), where results stabilized at around 69% accuracy for several years.
In this context, deep learning appears as a promising approach, particularly for the study of Music Emotion Variation Detection (MEVD), i.e., the analysis of emotion variations throughout a complete song.
Objetivo
The objectives to be achieved in this project are the following:
1. Research and development of deep learning (DL) approaches for static MER
2. Research and development of DL approaches for MEVD
Plano de Trabalhos - Semestre 1
1. State-of-the-art review: MER, deep learning for audio in general and MER in particular
2. Update of the current MER and MEVD datasets (previously created by our group)
3. Evaluation of the current algorithms (based on classical feature engineering and machine learning) on the novel datasets
Plano de Trabalhos - Semestre 2
4. Continuation of dataset updating
5. Research and development of deep learning (DL) approaches for static MER (e.g., via CNNs, transfer learning, etc.)
6. Research and development of DL approaches for MEVD (e.g., via LSTMs, etc.)
7. Critical evaluation of the developed DL models
8. Writing of a scientific paper and thesis
Condições
---
Observações
---
Orientador
Prof. Rui Pedro Paiva e Prof. Renato Panda
ruipedro@dei.uc.pt 📩