Extending the Elastic Microservices Platform
Áreas de especialidade
Engenharia de Software
Engenharia de Software
Local do Estágio
With increased reliance on the cloud and most modern systems being designed as cloud native, and given the trend in the industry to follow service oriented architectures, particularly microservices, there is renewed interest in creating modern modeling tools for performance analysis. The mentioned tools are particularly interesting for resource scheduling, to achieve optimal resource allocation or elastically scalable systems, meaning that resources are provisioned as necessary in response to demand, and deallocated when not required.
On complex, non-linear systems, deciding how to distribute resources is non-trivial. Ideally, resources should be allocated to the point of the system responsible for some bottleneck, for example, the service that causes the largest delay, or throttles throughput the most. Formal models, generated from logs or traces have been used with success to simplify the decision process and create useful visualizations.
In the previous work of the DataScience4NP (https://osf.io/8dc2e/) project, we developed a monitoring platform for legacy systems requiring no dedicated tracing. This platform resorted to the idea of an API gateway that could perform most monitoring operations transparently to applications. At the same time, we developed an elastic microservices platform (EMP), capable of providing automatic and elastic scaling of deployed applications. This platform departed from the state of the art, because existing services are proprietary and rely on (manual) user-made rules, to achieve their elastic and scaling capabilities. While in the API gateway approach we managed to create a monitoring tool, with the EMP such tool does not exist.
The purpose of this intership is to extend the Elastic Microservices Platform (EMP) of the DataScience4NP project. EMP should bring together our previous efforts regarding observability and control of fine-grained distributed systems based on microservices. For this, the intern student should bring under the same monitoring and control platform, tailor-made applications containing sufficient instrumentation and legacy applications that should run transparently without modifications. Based on the information collected and on high-level user-defined rules, the infrastructure should automatically adjust replication to meet performance demands. EMP should depart from state of the art solutions, where low-level user-made rules directly control the infrastructure.
Plano de Trabalhos - Semestre 1
- Study the state of the art, namely existing tools, and related technologies like tracing (2 months);
- Integrate the existing work (1 month);
- Define requirements of the monitoring tool (1 month);
- Write intermediate report (1 month).
Plano de Trabalhos - Semestre 2
- Implement the monitoring tool (2 months);
- Test and evaluate the results (2 months);
- Write final report (1 month);
This work should take place in the context of a research project funded by FCT. If the project gets an extension it may support a scholarship of 745 euros/month for 3 months with a possible extension.
Filipe Araújo, Prof. Rui Paiva e Prof. Jorge Cardoso (Huawei)