Titulo Estágio
Evolving Learning Rate Schedulers
Áreas de especialidade
Sistemas Inteligentes
Local do Estágio
CISUC
Enquadramento
Deep Artificial Neural Networks are one of the most components of modern Artificial Intelligence. These systems are able to achieve good results in several problems through their ability to autonomously learn how to create better solutions. While the learning process is largely automatic there is still a large degree of freedom in what concerns design decisions and parameters that must be manually tuned, e.g., what learning algorithm to used, and respective parameterisation. In this internship we will focus our attention on the learning rate. While the value of this parameter has a massive impact on the network's performance the user must use trial-and-error along with their own experience to manually find an adequate initial value and decaying strategy. There have been several proposal in the literature concerning the usage of automatic methods that use the information provided by the training to dynamically adjust the learning rate on the fly.
Objetivo
We will study the possibility of using Evolutionary Algorithms to evolve a Learning Rate Schedulers for a specific network architecture and comparing it to state of the art schedulers and optimisers.
Plano de Trabalhos - Semestre 1
• Review of the state of the art
• Analysis of the existing learning rate schedulers
• Proposal of 1st Version of an EA to Evolve Learning Rate Schedulers
• Documentation
Plano de Trabalhos - Semestre 2
- Proposal of 2nd Version of an EA to Evolve Learning Rate Schedulers
- Implementation and Validation of the proposed approach
- Experimental analysis
- Documentation
Condições
The work is to be conducted at the CMS and ECOS laboratories of CISUC. A workplace will be provided as well as the required computational resources.
Orientador
Nuno Lourenço / Penousal Machado
naml@dei.uc.pt 📩