Titulo Estágio
Dependability Analysis of Autonomous Robots
Áreas de especialidade
Engenharia de Software
Local do Estágio
CISUC-SSE
Enquadramento
Autonomous systems are currently one of the most interesting and promising research topics. Significant breakthroughs have been made in recent years, with various autonomous systems appearing in everyday tasks, from industry to agriculture and self-driving cars. As a result, work and research on this subject have grown considerably [1]. Modern autonomous systems are complex and comprised of several components and sensors that continuously monitor and interact with their environment. Additionally, most of these systems also leverage Machine Learning (ML) to integrate the various sources of data to make real-time decisions.
As autonomous systems become more common and are used in critical tasks, the dependability of such systems becomes of utmost importance. The development of such systems typically follows strict processes using various verification and validation techniques [2]. Notwithstanding, due to the increasing size and complexity of these systems, such techniques do not scale well and require considerable time and cost to achieve relevant coverage. As the goal of these systems is often to execute critical tasks or operate alongside humans, assessing their behavior under abnormal conditions is also necessary to mitigate potential risks or costs. Moreover, although ML has shown remarkable results, it has also been shown that specific inputs cause unexpected results. Assessing this on real systems would be impractical and nearly impossible, thus, various techniques have been developed over the years [3]. Fault injection is a technique that intentionally introduces faults in the system in an attempt to mimic real faults [4], with the purpose of assessing the behavior of the system. With it, it is possible to assess the impact that residual faults have on the system (e.g., fault tolerance, dependability validation).
Another specific characteristic is that there are various types of autonomous vehicles, from objective robots to self-driving cars and unmanned aerial vehicles. Additionally, contrary to typical software systems, autonomous systems imply physical components and hardware, and thus properly testing them is not straightforward. Real-life tests can be dangerous and expensive, hence, various simulators have been developed, each with specific considerations and goals (e.g., Gazebo, CARLA) [5]. As the interest in autonomous systems has been so pervasive, various tools and frameworks to support their development have been created (e.g., Robot Operating System (ROS), PX4). Such frameworks have become considerably popular with countless applications and large communities.
Each framework and tool allows different configurations and uses, that ultimately influence which techniques can be used to assess and explore the dependability of such systems. Notwithstanding, autonomous systems are a fairly new research field, and existing literature on assessing and improving their dependability is still scarce and fragmented. Ultimately, it is difficult to identify which verification and validation techniques are commonly used and which development frameworks and tools support which techniques.
The focus of this internship is to identify and thoroughly explore and detail which techniques can be used to assess the dependability of autonomous systems, from hardware and sensors to the ML algorithms and models used to make real-time decisions, with a special focus on fault injection techniques. The goal is to conduct a systematic analysis of existing related work on the use of dependability techniques on the various tools and frameworks available for the development of autonomous vehicles. This analysis should highlight how the various techniques integrate with the respective tools and frameworks and what kind of dependability assessment they allow on the various components of the system. Afterward, a proof-of-concept on using fault injection on a selected set of tools should be implemented to demonstrate how fault injection can be used to improve the dependability of autonomous systems.
1. Jahan, F., Sun, W., Niyaz, Q., & Alam, M. (2019). Security modeling of autonomous systems: a survey. ACM Computing Surveys (CSUR), 52(5), 1-34.
2. ZZJha, S., Banerjee, S. S., Tsai, T., Hari, S. K., Sullivan, M. B., Kalbarczyk, Z. T., ... & Iyer, R. K. ML-based Fault Injection for Autonomous Vehicles.
3. Avizienis, A., Laprie, J. C., Randell, B., & Landwehr, C. (2004). Basic concepts and taxonomy of dependable and secure computing. IEEE transactions on dependable and secure computing, 1(1), 11-33.
4. Arlat et al., 1990
5. Rosique, F., Navarro, P. J., Fernández, C., & Padilla, A. (2019). A systematic review of perception system and simulators for autonomous vehicles research. Sensors, 19(3), 648.
Objetivo
The learning objectives of this master’s internship are:
1) Dependability, fault tolerance: study the subject of fault tolerance and its techniques as means to improve the dependability of modern systems;
2) Fault Injection: understand how to use fault injection and conduct representative fault injection campaigns to assist in developing dependable systems;
3) Autonomous systems: study the challenges and development methods and tools used to develop autonomous systems;
4) Machine Learning: study and understand how ML models can be used to support decision-making in autonomous systems and how their dependability can be assessed;
5) Research Design: understand how to design and execute an experimental process to address complex and open research issues.
Plano de Trabalhos - Semestre 1
[12/09/2022 a 20/10/2022] Literature review
Study the concepts to be used in the internship, namely autonomous systems and existing development techniques and tools, fault tolerance, fault injection,
[21/10/2022 a 30/11/2022] Identify existing development tools for autonomous systems
Systematically identify existing development tools (e.g., frameworks, simulators) available to support the development of autonomous systems. Highlight relevant characteristics, goals, advantages, and disadvantages.
[01/12/2021 a 15/01/2023] Write the dissertation plan
Plano de Trabalhos - Semestre 2
[06/02/2023 a 6/03/2023] Revise existing literature on assessing the dependability of autonomous systems
Thoroughly assess and review existing works on assessing the dependability of autonomous systems, with a focus on using fault injection. This includes exploring and understanding how the different works integrate with the architecture of each tool and what kind of assessment the various approaches provide, from hardware to ML models.
[7/03/2023 a 17/04/2023] Devise an experimental process to create a proof-of-concept of using fault injection to assess the dependability of autonomous vehicles
Select a subset of tools used for developing autonomous systems and devise an experimental process on using fault injection to assess the dependability of the system. To ensure the representativeness of the process this includes properly identifying where the faults can be injected, defining the fault model, and the failure modes. This also requires modeling a world/scenario where the robot will operate, and a set of tasks it must perform.
[18/04/2023 a 08/05/2023] Implement the proof-of-concept
Implement the experimental process using the selected techniques. Process, explore, and analyze the results of the experiments to understand the behavior of the target system in the presence of faults. Compare with failure data from existing works
[09/05/2023 a 05/06/2022] Write the thesis.
Condições
Depending on the evolution of the internship a studentship may be available to support the development of the work. The work is to be executed at the laboratories of the CISUC’s Software and Systems Engineering Group. A workplace will be provided as well as the necessary computational resources.
Observações
Trabalho a ser co-orientado por João Pedro Gonçalves Teixeira de Macedo e Lino José Forte Marques
Orientador
João R. Campos
jrcampos@dei.uc.pt 📩