Titulo Estágio
IMDB - Intelligent Multimodal Detection of Biodiversity
Local do Estágio
DEI, CISUC
Enquadramento
The world is reaching its resource usage limits earlier each year. To address this, humans should adopt better practices to reduce our footprint and carbon dioxide emissions. These practices include energy-saving measures, water conservation, making sustainable dietary choices, choosing sustainable clothing, and opting for eco-friendly transportation. Adopting these practices can significantly contribute to mitigating our impact on the planet's limited resources.
The agricultural sector, a significant contributor to the human footprint, should shift towards practices that are kinder to the climate and the planet. Landscapes that foster biodiversity, as opposed to extensive urban development, are indicative of a healthier ecological state. An area rich in biological diversity suggests minimal environmental impact, in contrast to urban areas where the prevalence of vehicles and buildings often leads to diminished natural spaces.
In this internship, we propose a system that leverages Deep Neural Networks not only for detecting animal biodiversity in images, videos, or sound but also for a wider scope of environmental assessment. This includes classifying various species and identifying elements that contribute to or detract from ecological health. Our research group has previously focused on identifying insects in traps in natural settings using State-of-The-Art algorithms, such as, You Only Look Once (YOLO) and Region-based Convolutional Neural Networks (R-CNNs). Regarding detection through sound, established algorithms, like BirdNet, already effectively identify birds. The aim of this internship is to expand intelligent detection to a broader range of animal species and identify factors detrimental to healthy environments, such as engine noises and urban sounds, or imagery of vehicles and urban areas. By doing so, we can assess the environmental health of a location more accurately. The challenge of this internship is to build a multimodal system that integrates various types of data, such as images and sounds, to effectively detect biodiversity factors.
In agriculture and production, there is a growing trend of managing practices by balancing CO2 emissions through carbon credits. The future extends beyond just balancing CO2 credits; it includes biodiversity credits as well. This approach is perfectly aligned with the future trajectory of our society and its evolving standards.
Objetivo
In this internship, the student is expected to study, propose, implement, and test models for the intelligent detection of biodiversity elements as well as urban elements, using both images and sounds. To achieve this goal, the following objectives will be pursued:
- Study the state of the art
- Study the available datasets, namely the tomato white fly dataset (Nieuwenhuizen 2018) and the bird net dataset (kaggle) together with datasets form the project associated with this proposal.
- Study the available frameworks for model development and data integration
- Define, implement, and fine tune the learning architecture
- Propose and deploy test setup
Plano de Trabalhos - Semestre 1
- Literature review
- Identification and annotation of datasets for model development
- Identification and study of available frameworks for model development
- Define the architecture of the system
- Start implementing the proposed approach
- Write intermediate report
Plano de Trabalhos - Semestre 2
- Implement the proposed solution and fine tune models
- Test and evaluate performance
- Write final report
Condições
This work should take place in the context of a research project funded by PRR. There is the possibility of a 6-month scholarship.
Orientador
Catarina Silva, Rui Paiva, Dinis Costa
catarina@dei.uc.pt 📩