Titulo Estágio
Using generative AI to tailor maturity models
Áreas de especialidade
Engenharia de Software
Sistemas de Informação
Local do Estágio
DEI
Enquadramento
Establishing and sustaining competitive advantage is a key organizational objective. Historically, maturity models have facilitated this endeavour by guiding process improvement initiatives [1]. Studies indicate a positive correlation between the application of such models, process enhancement, and resulting business performance [2].
Nevertheless, the efficacy of maturity models is also challenged. A key criticism, highlighted by Forsgren [3], concerns their prescriptive nature, which often assumes organizational stasis and overlooks the dynamic, context-specific factors influencing growth. This 'one-size-fits-all' approach, coupled with the implicit 'end state' of perfection in many models, can impede rather than foster continuous improvement.
In response, some models, such as COBIT [4] and CMMI [5], began including mechanisms to tailor the maturity model to the goals and characteristics of the organization where it will be applied. However, these often consist of a set of manual steps that the practitioners perform. In CMMI, this involves selecting from a set of standard processes that can be implemented differently in the context of a project or a service. In COBIT, the practitioner answers multiple questions from the organization to select and adapt the practices that will be employed.
Our objective is to assess whether a Large Language Model (LLM) can be used to perform this process. As a starting point, we’ll analyse the following LLMs: GPT-4o, Gemini, and LlaMA 3.1.
This experiment will be conducted using a next-generation maturity model being developed at CISUC that currently addresses cybersecurity maturity. As part of this work, the student will have access to the next-generation maturity model with all its definitions, practices, and usage data from two different companies.
References:
1. T. De Bruin, M. Rosemann, R. Freeze, and U. Kaulkarni, “Understanding the Main Phases of Developing a Maturity Assessment Model,” Australasian Conference on Information Systems (ACIS), 2005
2. R. K. Kandt, “Experiences in improving flight software development processes,” IEEE Softw, vol. 26, no. 3, pp. 58–64, 2009, doi: 10.1109/MS.2009.66
3. N. Forsgren, J. Humble, and G. Kim, Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations. IT Revolution Press, 2018.
4. COBIT | Control Objectives for Information Technologies | ISACA. (n.d.). https://www.isaca.org/resources/cobit
5. CMMI Institute. (2023). CMMI - Build capability, Drive performance. ISACA - CMMI Performance Solutions. https://cmmiinstitute.com/
Objetivo
The main objectives of this work are:
• Select the most adequate LLM for supporting the development of the Generative AI Agent
• Develop a Generative AI Agent to tailor a maturity model to a specific organization or team
• Apply the Generative AI Agent in different scenarios to validate it
Plano de Trabalhos - Semestre 1
Plano de Trabalhos Semestre 1:
1) Research the state-of-the-art on the initial set of LLMs and their use for analyzing process improvement data.
Start: September 2025
Duration: 2 months
2) Research Generative AI Agents architectures
Start: November 2025
Duration: 1 month
3) Define the requirements and architecture for the Generative AI Agent
Start: December 2025
Duration: 1 month
Plano de Trabalhos - Semestre 2
Plano de Trabalho Semestre 2:
4) Implement the Generative AI Agent
Start: January 2026
Duration: 3 months
5) Test the Generative AI Agent with pilot companies
Start: April 2026
Duration: 1 month
6) Write the thesis and a scientific paper
Start: May 2026
Duration: 2 months
Condições
The student will be integrated in the Information Systems group (ISG) of CISUC, Department of Informatics Engineering, University of Coimbra. A workplace and the required resources will be provided.
Observações
Supervisors for this work are:
Nuno Seixas: researcher at the Centre for Informatics and Systems of the University of Coimbra, Portugal. His research interests are in Software Engineering, Maturity Models, and software engineering team optimizations. He is pursuing his PhD in this area at the University of Coimbra. Nuno is also an Invited Assistant Professor at the Department of Informatics Engineering of the University of Coimbra, teaching Software Engineering classes. Additionally, Nuno has over 20 years of experience in the software engineering industry, having worked in healthcare informatics, telecommunication, and other domains. With extensive experience working with process improvement tools, he is also a Certified Instructor for CMMI-DEV. Contact him at naseixas@dei.uc.pt.
Paulo Rupino da Cunha: Associate Professor of Information Systems with Habilitation and former head of the IS Group at the Faculty of Science and Technology of the University of Coimbra, Portugal. Paulo holds a Ph.D. (2001) in Informatics Engineering from the University of Coimbra. He has been Adjunct Associate Teaching Professor in the School of Computer Science at Carnegie Mellon, USA, and Visiting Associate at Brunel University, UK. Serves in the editorial board of various journals and has published more than 160 papers, including in A/A* conferences such as the International Conference on Information Systems (ICIS), European Conference on Information Systems (ECIS), Americas Conference on Information Systems (AMCIS), Hawaii International Conference on System Science (HICSS), International Conference on Information Systems Development (ISD), and in top journals such as Electronic Markets, Business & Information Systems Engineering, Requirements Engineering, Information Systems Development, Journal of Enterprise Information Management, and Information Technology for Development, among others. Contact him at rupino@dei.uc.pt.
Marco Vieira: professor of computer science at the University of North Carolina in Charlotte. He earned his Ph.D. in Informatics Engineering from the University of Coimbra, Portugal. His research focuses on dependability and security assessment, fault injection, and software testing, with over 250 publications. He chairs IFIP WG 10.4 on Dependable Computing and Fault Tolerance, serves as Associate Editor for IEEE TDSC, and holds leadership roles in DSN, ISSRE, and LADC. He has coordinated multiple national and international research projects in his field. Contact him at marco.vieira@charlotte.edu.
Orientador
Paulo Rupino da Cunha
rupino@dei.uc.pt 📩