Titulo Estágio
Model Governance Framework for Auditable ML Deployment Nome da
Local do Estágio
Remote, Hybrid, or Lisbon/Coimbra/Porto offices
Enquadramento
In high-stake systems, such as financial institutions, ML models are bound to comply with certain expectations; therefore, tooling for auditing and model evaluation is essential to foster transparency and trust around model decisions.
This work focuses on developing visualization techniques for model governance throughout the model lifecycle: from training to production.
Objetivo
● Generate Model Governance report upon new model training or refresh.
● Enhance the Model Metadata Database to include all necessary data to audit a model
● Develop a Kubernetes service to create model governance visuals automatically whenever a model is deployed
● WebUI to consult Model Governance reports in an interactive way.
● Understand user needs and requirements around Model Governance.
● Review previous work done at Feedzai to create visualizations that transparently convey information about model decisions to end users
Plano de Trabalhos - Semestre 1
● Understand Context and Problem Statement
○ Deep dive into general ML model concepts (Precision, Recall, FPR, ROC curves) and tree-based models (LightGBM, XGBoost, Random Forests)
○ Introduction to Fraud Detection domain
○ Interview Data Scientists to understand expectations around Model Governance Reporting
● Literature Review
○ Explore state-of-the-art techniques to provide Model Auditability and Transparency and visualization techniques.
○ Investigate specific open-source libraries or frameworks that aid in model auditability and transparency.
○ Research methods for quantifying and visualizing model uncertainty.
● Tech Set up and Planning
○ Be comfortable with Feedzai tech stack: kubernetes, quarkus, python, React, TypeScript, etc.
○ Design the architecture of the model governance and evaluation service.
○ Set up local development environment
Plano de Trabalhos - Semestre 2
Plano de Trabalhos do 2º. Semestre (Tempo inteiro 40h/s):
● Prototype Development
○ Define data requirements for Model Governance Service
○ Extend the Model Metadata Database to include necessary fields for previous step
○ Develop a Kubernetes service to create model governance visuals automatically whenever a model is trained and deployed
○ WebUI Interface on Feedzai Risk Studio to consult Model Governance reports in an interactive way.
○ Investigate best practices and visualizations to improve trust and transparency on model decisions.
○ Propose set of visualization and client parametrization options
● Write Thesis and Internal Reports
○ Write main thesis document
○ Develop a user guide for stakeholders who will use Model Governance Framework.
○ Present project and its outcomes in various forums, including Research Show&Tell
Condições
Students may work remotely or from Feedzai’s offices in Lisbon, Porto or Coimbra (hybrid models supported). We offer flexible working hours. During the second semester, students will have a paid contract with Feedzai with access to Feedzai laptop and Feedzai internal platforms, infrastructure and proprietary datasets.The student will receive dedicated mentorship by Feedzai researchers, with regular one-on-one check-ins and weekly team syncs. All scientific contributions will be acknowledged in patents and publications where applicable.
Observações
n/a
Orientador
Cláudio José Pereira Correia
claudio.correia@feedzai.com 📩