Titulo Estágio
Mixed Reality Videoconferencing for 3D Object Inspection
Áreas de especialidade
Comunicações, Serviços e Infraestruturas
Sistemas de Informação
Local do Estágio
DEI
Enquadramento
Mixed Reality (MR) merges real and virtual worlds, allowing the interaction between physical and digital objects in real time. While Virtual Reality (VR) creates artificial environments where users are completely immersed and Augmented Reality (AR) overlays digital information on the real world in a partially immersive experience, Mixed Reality extends the concept of the latter by enabling a deeper interaction between real and virtual worlds. In a MR environment, real and digital elements interact between them and with the user, providing a more complete experience. Recently, the concept of eXtended Reality (XR) has been used to refer to any experience in the spectrum AR-MR-VR.
In this work, the inspection of 3D models between two parties connected through an XR enhanced videoconference will be explored.
The use of videoconferencing tools in business meetings has been increasing steadily in recent years, and sharply since the beginning of COVID-19, as the travel restrictions caused by the pandemic lead to record levels of growth. However, the current videoconferencing tools are not suitable to replace all meeting needs.
In this project, we are interested in developing a videoconferencing tool that allows participants to use an immersive VR headset such as the Oculus Quest 2 to participate in a videoconference with other participants (also using a VR headset or their desktop computers). The videoconferencing tool should allow participants to discuss, inspect, manipulate, and annotate 3D models, in the context of a high precision mold industry.
Objetivo
The work to be done under this proposal has three main objectives:
- Elicitation of requirements for the functional characteristics of the immersive and non-immersive modes of the videoconferencing tool (what should users be able to view and manipulate in a shared space, in both immersed and non-immersed settings; what should users view and manipulate in a personal space; how should be immersed user be represented to the non-immersed users?)
- Definition of the characteristics of the data model for shared space (what properties do we need to keep synchronized between the multiple participants) and implementation of the server-side mechanisms to provide this shared data model
- Development of a videoconferencing tool that allows 3D model inspection and discussion that combines standard video and audio communication, with shared 3D model manipulation.
In previous projects, we have used the OpenVidu videoconferencing platform as the main video and audio communication tool. This project should ideally take advantage of this platform.
Plano de Trabalhos - Semestre 1
1) Study of the state-of-the-art of XR videoconferencing/collaboration tools with focus on immersive tools
2) Study of videoconferencing development platforms (including OpenVidu) with focus on adding realtime communication of non-media data for keeping and communicating the state of the various participants in the session.
3) Define the requirements and make prototype of the videoconferencing tool
4) Define how the system should be evaluated
5)Write intermediate report
Plano de Trabalhos - Semestre 2
6) Development of a working prototype of videoconferencing tool that supports the use of 3D models in a Mixed Reality enhanced videoconference
7) Tests and validation
8) Writing of the final report
Condições
The work will be developed at CISUC. Student will have access to an Oculus Quest 2 device.
Observações
An occasional visit to a company engaged in the production of technical molds of high precision and performance, may be required.
Orientador
Jorge Carlos Santos Cardoso / Vasco Nuno Sousa Simões Pereira
jorgecardoso@dei.uc.pt 📩