Titulo Estágio
Towards Physical Plausibility in Neuroevolution Systems
Áreas de especialidade
Sistemas Inteligentes
Local do Estágio
CMS - CISUC
Enquadramento
Current state-of-the-art deep ML models are highly inefficient energy-wise. For instance, running GPT3 requires 5 000 000 watts of energy, and training requires even greater power. These energy costs make Deep ML inaccessible to most individuals and companies (even large ones), hindering its applicability, and making it energetically unsustainable on a global scale. In contrast, the human brain consumes, roughly, 30 watts.
Our main goal with this project is to promote the evolution of efficient Artificial neural network architectures by looking at the main source of efficient intelligence, i.e. nature, and by taking into account a series of constraints that may have promoted the evolution of our brains, including pressure for energy efficiency, physical plausibility and learning adeptness.
Objetivo
We will address the problem of creating Energy-Efficient Neural Architecture. To address it we will build upon previous research of the group in this area, namely DENSER, to promote the evolution of physically plausible architectures. Fully connected layers are an integral part of most, if not all, ANNs, However, their profusion is biologically and physically infeasible. Moreover, there is reason to believe that fully connected models are counterproductive, in the sense that the emergence of specialized modules, located in specific and contained regions of the brain may be a byproduct of the restrictions that exist in nature, which indirectly force the emergence of modular and hierarchical structures.
Plano de Trabalhos - Semestre 1
T1 – State of the art on EML
T2 – Familiarization with DENSER
T3 – Proof of concept / Early Prototype
T4 – Framework proposal
T5 – Writing of the Intermediary report
Plano de Trabalhos - Semestre 2
T1 – Implementation of the Framework
T2 – Experimentation and validation
T3 – Refinement
T4 – Article writing and submission
T5 – Writing the final report
Condições
The work will be conducted in the Cognitive and Media Systems (CMS) and Evolutionary and Complex Systems (ECOS) groups from CISUC.
The student is awarded a scholarship (Bolsa de Investigação para Licenciado) for at least 6 months, renewable for an equal period by agreement between the advisor and the intern.
Orientador
Penousal Machado / NUno Lourenço
machado@dei.uc.pt 📩