Titulo Estágio
An Explainable AI (XAI) Tool for Diverse Machine Learning Models
Local do Estágio
Instituto Pedro Nunes, Coimbra (Laboratório de Informática e Sistemas)
Enquadramento
Explainable AI (XAI) techniques (1), such as LIME, SHAP, attention mechanisms, and Layer-wise Relevance Propagation (LRP)(2) , have significantly advanced our understanding of the decision-making processes of complex machine learning models (3) . However, these existing tools are often domain-specific, limiting their adaptability to different scenarios.
The objective of this thesis is to research and develop a use case-agnostic XAI tool with a modular architecture. This tool should facilitate the integration with different machine learning models. The tool should support algorithm agnosticism and flexible input handling. Context awareness, feedback mechanisms, and counterfactual explanations should also be considered. These features aim to enhance the transparency and performance of different AI models on diverse applications and help address the current limitations of XAI tools and advance towards more reliable and trustworthy AI systems.
This topic is part of the NEXUS project, in which Pedro Nunes Institute (IPN) is a consortium member.
(1) Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why Should I Trust You?": Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
(2) Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015). On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation. PLOS ONE
(3) Lundberg, S. M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions.
Objetivo
The primary objective of this thesis is to design and develop a flexible XAI tool that offers tailored insights through adaptable inputs, algorithm agnosticism, and context awareness. The specific objectives of this master thesis are as follows:
• Review and Analyse Existing XAI Techniques - The first step of the work is to perform a comprehensive literature review of current XAI techniques while focusing on LIME, SHAP, attention mechanisms or Layer-wise Relevance Propagation (LRP)) and respective applications. The state-of-the-art review should also identify the limitations of existing XAI tools in different domains (e.g., LIME's difficulty with image data, SHAP's computational complexity for large datasets).
• Design a Modular XAI Tool – After the state-of-the-art analysis, the student should define the architecture of the XAI tool that allows seamless integration with various machine learning models and datasets (e.g., modular plugins for integrating with TensorFlow and PyTorch models). The tool should also support flexible input handling, supporting, for instance, structured data in XGBoost (4) and unstructured data in BERT (5).
• Support Advanced Features – Building on the proposes prototype, the solution should support 1) context awareness inputs to enhance the relevance and accuracy of explanations (e.g., user behaviour data for personalization in recommendation systems, environmental factors in predictive maintenance models); 2) feedback mechanisms to continuously improve the tool's performance (e.g., user feedback loops or iterative refinement based on domain expert input); and 3) an interface for counterfactual explanations to provide actionable insights (e.g., altering feature values to see different classification outcomes, understanding changes needed for positive or negative outcomes).
• Evaluate and Optimize the Tool – The development work should be complemented with experimental work to assess the tool's performance and adaptability using diverse machine learning models and datasets (e.g., evaluating on sentiment analysis with BERT, image classification with ResNet (6) , or regression tasks with XGBoost). The optimization process should also include user studies on the interpretability of model explanations or quantitative measures of explanation accuracy in order to assess the meaningfulness of the explanations generated by the tool.
• Provide Recommendations – One of the outcomes of this thesis is to provide practical guidelines or instructions for the deployment and integration of the proposed XAI tool in environments such as Kubernetes or Docker.
By the end of this research, the student is expected to develop a flexible XAI tool that can be easily adapted to various machine learning models and datasets, providing clear and actionable insights while addressing current challenges in explainability. This work will contribute significantly to the field of explainable AI and offer valuable tools for entities seeking to enhance the transparency and reliability of their AI systems.
(4) https://github.com/dmlc/xgboost
(5) https://huggingface.co/google-bert/bert-base-uncased
(6)https://www.tensorflow.org/api_docs/python/tf/keras/applications/ResNet50
Plano de Trabalhos - Semestre 1
By the end of this research, the student is expected to develop a flexible XAI tool that can be easily adapted to various machine learning models and datasets, providing clear and actionable insights while addressing current challenges in explainability. This work will contribute significantly to the field of explainable AI and offer valuable tools for entities seeking to enhance the transparency and reliability of their AI systems.
Work plan, first semester:
[Weeks 1-4] - Literature review on XAI techniques and identification of limitations in existing tools.
[Weeks 5-8] - Research and analyse context-awareness techniques for enhancing XAI tools.
[Weeks 9-12] - Design the modular architecture for the XAI tool, ensuring flexibility and algorithm agnosticism.
[Weeks 13-15] - Implement and evaluate initial prototypes of the XAI tool using test models and datasets.
[Week 16-20] – Prepare first intermediate report.
Plano de Trabalhos - Semestre 2
Work plan, second semester:
[Weeks 1-6] - Refine and enhance the XAI tool based on initial evaluations.
[Weeks 7-10] - Perform comprehensive experiments using various machine learning models and datasets.
[Weeks 11-15] - Analyse the experimental results, compare with existing XAI tools.
[Week 16-20] - Finalize the master's thesis report, submission of document and preparation for final thesis defence.
Condições
The workplace will be at the Instituto Pedro Nunes (IPN) Computer and Systems Laboratory.
This work will be part of a research project. Upon a successful first semester, the student may apply for a research grant for a graduate, according to the IPN's scholarship regulations approved by FCT, for a period of up to 6 months, possibly renewable.
Observações
During the application phase, doubts related to this proposal, namely about the objectives and conditions, must be clarified with the supervisors, via email or a meeting, to be scheduled after contact by email.
Orientador
Paulo Miguel Guimarães da Silva
pmgsilva@ipn.pt 📩