Titulo Estágio
BERT With Adapters For Chat2Trade
Local do Estágio
Hybrid: Remote + BNP Paribas Colombo Office - R. Galileu Galilei 2, 1500-392 Lisboa, Portugal
Enquadramento
BNP Paribas’ Global Markets (GM) department offers a broad range of products and services in the global interest rates, credit, currency, equities & commodity markets to help their franchise of client find effective ways to raise and invest capital as well as manage their exposure to risk. It has scale and reach to conduct business anywhere in the world and deliver products denominated in almost all of the world’s currencies.
The Data and Artificial Intelligence Lab is a team created 6 years ago to leverage the latest techniques of AI and Machine Learning for the benefit of the whole Global Market business. The team is composed by 30 data scientists, spread across the world with people in London, Paris, Lisbon, Singapore, Frankfurt and New York. The team's mandate is to develop machine learning based solutions for the business and currently we do so by tackling natural language processing, time series prediction and strategy optimization problems.
Objetivo
Apply transfer learning with adapter modules. In NLP, fine-tuning large pre-trained language models is an effective transfer mechanism that achieves great performance in many downstream tasks. But the higher the number of tasks, more parameter inefficient the fine-tuning process becomes, since an entire new model is required for each task. We want to assess the effectiveness of using adapter modules to perform those same tasks. Adapter modules yield a compact and extensible model, since they add only a few trainable parameters per task, and new tasks can be added without revisiting previous ones. The parameters of the original network remain fixed, yielding a high degree of parameter sharing. The goal is to apply this transfer mechanism and study its impact on performance.
Plano de Trabalhos - Semestre 1
Research and concrete definition of:
- State of the art;
- Project mandate, objectives and baselines;
- Techniques, methodologies and computational tools to use.
Familiarization with the working environment and available tools and resources.
Planning of the work to develop in the following semester.
Plano de Trabalhos - Semestre 2
Following the work planning elaborated in the 1st semester, the student will tackle:
1) Implementation
2) Evaluation
3) Integration
of the proposed solution.
Before step (3), where we expect the solution to be integrated with the team’s toolkit and codebase, there might be the need for several iterations of the development cycle, where the student experiments with new techniques, ideas or approaches, assesses their impact and either moves on or goes back depending on the results.
The internship finalizes with a presentation of the developed work to the rest of the team.
Condições
The student might work remotely or in the Lisbon office and will be paid 1150/month for 3.5 months (or the equivalent spread out by 9 months).
Additional conditions might be given to the student.
Observações
[1] – BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
[2] – Parameter-Efficient Transfer Learning for NLP
- Regarding the internship place, the 1st semester might be full remote if need-be while the 2nd semester is preferably at BNP Paribas premises.
- The work will be conducted by 2 interns, working together under the team’s supervision.
Orientador
Julien Dinh
julien.dinh@bnpparibas.com 📩