Titulo Estágio
Automatically identify infrastructure problems for CI/CD pipelines using AI
Áreas de especialidade
Engenharia de Software
Sistemas Inteligentes
Local do Estágio
Fully Remote @Scalabit
Enquadramento
Continuous Integration (CI) and Continuous Delivery (CD) are software development practices aimed at automating the process of building, testing, and deploying software. Usually this is done using pipelines on top of tools like Jenkins, GitHub, or any other popular development platform.
To achieve CI/CD the developers write a pipeline that will automate all the tasks required to qualify and release their software. So every time that a developer pushes his/her changes to the remote then a pipeline is automatically triggered and starts the qualification. Repositories with a lot of collaboration can have pipelines running thousands of times per day. This is the safety net that allows you to release your software in a fully qualified and automatic way.
It’s quite often to find pipelines that fail. Failures lead to time investigating what is the nature of the failure. The nature of the failure can be between other things, a mistake from the developer or just some infrastructure failure. Diagnosing these failures is a task that requires looking at different logs and finding out what changed. With the current developments in AI we believe that a machine would do this job better than any human.
Objetivo
This internship has the following goals:
- Understand the current state-of-the-art around CI/CD tools
- Extract logs from github/gitlab pipelines (Scalabit will provide the repositories) and workout the data to be used in machine learning models (data is publicly available and can be extracted using an API)
- Create a machine learning model that can identify when a pipeline fails and what is the root cause of failure
- Give suggestions in case of failures on what can be improved
- Retrieve metrics from existing pipelines and identify what are the types of failures
Plano de Trabalhos - Semestre 1
- State of the art (Months 1 and 2)
The first stage will consist in studying background knowledge on the topics related to the thesis. Namely, knowledge on Continuous Integration, Continuous Delivery, state-of-the-art in terms of machine learning oriented to CI/CD pipelines. The student is expected to explore and collaborate in the software community to better understand the problem at hand. At this point, the thesis chapter on the state of the art shall be drafted.
- Preparation of a case study and initial version of the machine learning model (Months 3 and 4)
Start working on the machine learning model based on the research done on the state of the art. Train the existing model and start running the first experiments in an existing repository.
- Intermediate report (Month 5)
The tasks carried out during the first semester will be documented in the form of an intermediate report, followed by a public presentation and discussion. The most relevant topics at this stage are: context, problem statement, state of the art and preliminary discussion of the solution and its intended objectives.
Plano de Trabalhos - Semestre 2
- Further optimization of the machine learning model (Months 6 and 7)
- Use the model in other repositories and start classification of type of errors (Month 8)
- Master’s thesis (Month 9)
The writing of the master's dissertation must be completed and the respective public presentation prepared. The dissertation must document all the work carried out, proposed solution, the results and the conclusions obtained.
Condições
The work will be carried out remotely, with the support of Scalabit for any computational resource required to carry out the work (e.g. cloud infrastructure).
Scalabit has a budget to pay the internship student which is discussed with the applicants
Orientador
Andre de Brito Passos
andre.passos@scalabit.dev 📩