Titulo Estágio
Service Discovery in Data Science Microservices
Áreas de especialidade
Engenharia de Software
Sistemas Inteligentes
Local do Estágio
DEI-FCTUC
Enquadramento
The leading consulting company McKinsey estimates that there will be a shortage of data scientists to enable organizations to explore the full potential of big data. By 2018, the United States alone will face a shortage of 140,000 to 190,000 professionals with strong analytical skills with the know-how to analyze big data to make effective decisions. This shortage will be more dramatic in Portugal since, in contrast to US universities that provide Data Science degrees for several years (e.g., at Berkeley and Carnegie
Mellon University), Portuguese universities are just making the first steps.
This shortage of professionals cannot be mitigated easily, since training students to become data scientists requires time and resources to teach skills from diverse knowledge areas such as Computer Science, Statistics, Business, and Data Visualization.
Hence, the objective of the FCT DataScience4NP project is to explore the use of visual programming paradigms to enable non-programmers to be part of the Data Science workforce. More specifically, the objective of the DataScience4NP project is to build Cloud Native Applications (CNA) for Data Science using microservices.
Objetivo
This thesis will develop one of the components of the DataScience4NP platform and will build on top of work already conducted in the scope of the project. Namely, the main goal is to develop a service registry and discovery for Analytics-as-a-Service (AaaS) services using technologies such as Docker and Kuberneters, Spring Cloud and the semantic descriptions Linked USDL and LSS USDL.
Microservice applications can have hundreds of services exposed as API to clients. In theory, clients can make requests to microservices directly. There are challenges and limitations of this approach: knowing all endpoints addresses; perform HTTP request for each peace of information separately; merge the result on a client side. A better approach is to use service registry and discovery with an API gateway: a single entry point to handle requests by routing them to the appropriate backend microservice or by invoking multiple backend services and aggregating the results.
Specific Objectives:
• To develop an API Gateway service
• To extend Linked USDL and LSS USDL for service discovery
• To implement a service registry and discovery module using machine learning techniques
Technologies:
• Kubernetes, Docker, and Spring Cloud
• Netflix Eureka and Zuul
• Linked USDL and LSS USDL
Plano de Trabalhos - Semestre 1
1st semester
- Review of the state of the art and technologies on service registry and discovery
- Requirement analysis (including both functional and non-function requirements)
- System architecture
- Writing of the preliminary thesis
Plano de Trabalhos - Semestre 2
2nd semester
- System development
- System testing
- Writing of the final thesis
Condições
The student might receive a scholarship from the FCT DataScience4NP project (745€ / month).
Orientador
Rui Pedro Paiva, Filipe Araújo e Jorge Cardoso
ruipedro@dei.uc.pt 📩