Titulo Estágio
Mixed Reality Videoconferencing for 3D Object Inspection
Áreas de especialidade
Comunicações, Serviços e Infraestruturas
Sistemas de Informação
Local do Estágio
DEI
Enquadramento
Mixed Reality (MR) merges real and virtual worlds, allowing the interaction between physical and digital objects in real time. While Virtual Reality (VR) creates artificial environments where users are completely immersed and Augmented Reality (AR) overlays digital information on the real world in a partially immersive experience, Mixed Reality extends the concept of the latter by enabling a deeper interaction between real and virtual worlds. In a MR environment, real and digital elements interact between them and the user, providing a more complete experience. Recently, the concept of eXtended Reality (XR) has been used to refer to any experience in the spectrum AR-MR-VR.
The use of videoconferencing tools in business meetings has been increasing steadily in recent years, and sharply since the beginning of COVID-19, as the travel restrictions caused by the pandemic lead to record levels of growth. However, the current videoconferencing tools are not suitable to replace all meeting needs.
In this work, we would like to explore a specific videoconferencing scenario where two or more users are inspecting and discussing 3D models, using a combination of XR devices (e.g. Microsoft HoloLens) and standard desktop computers. The tool should allow participants to discuss, inspect, manipulate, annotate, etc., 3D models, in the context of a high precision mold industry.
Objetivo
The work to be done under this proposal has three main objectives:
- Elicitation of requirements for the functional characteristics of the immersive and non-immersive modes of the videoconferencing tool (what should users be able to view and manipulate in a shared space, in both immersed and non-immersed settings; what should users view and manipulate in a personal space; how should be immersed user be represented to the non-immersed users?)
- Definition of the characteristics of the data model for shared space (what properties do we need to keep synchronized between the multiple participants) and implementation of the server-side mechanisms to provide this shared data model
- Development of a videoconferencing tool that allows 3D model inspection and discussion that combines standard video and audio communication, with shared 3D model manipulation.
In previous projects, we have used the OpenVidu videoconferencing platform as the main video and audio communication tool. This project should ideally take advantage of this platform.
Plano de Trabalhos - Semestre 1
1) Study of the state-of-the-art of XR videoconferencing/collaboration tools with focus on immersive tools
2) Study of videoconferencing development platforms (including OpenVidu) with focus on adding realtime communication of non-media data for keeping and communicating the state of the various participants in the session.
3) Study the development tools and platform for the XR device
4) Define the requirements and make prototype of the videoconferencing tool
5) Define how the system should be evaluated
6) Write intermediate report
Plano de Trabalhos - Semestre 2
7) Development of a working prototype of videoconferencing tool that supports the use of 3D models in a Mixed Reality enhanced videoconference
8) Tests and validation
9) Writing of the final report
Condições
The work will be developed at CISUC. Student will have access to a Microsoft Hololens device.
Orientador
Jorge C. S. Cardoso / Vasco Pereira
jorgecardoso@dei.uc.pt 📩