Titulo Estágio
Magnetic Interaction Toolkit for VR Applications
Áreas de especialidade
Sistemas de Informação
Engenharia de Software
Local do Estágio
DEI
Enquadramento
Mobile device-based VR systems are increasingly popular due to their low cost and ease of use. Headsets for attaching our personal smartphones are cheap (e.g. Google’s CardBoard project is a low cost, low quality cardboard device that costs about 15 dollars and that anyone can assemble at home) and can be used almost anywhere since it requires no cabling to external computers.
However, mobile based VR systems have an input problem: most controllers for VR input are designed for desktop systems and don't work with smartphones. Additionally, in this kind of VR experiences we are usually interested in low cost solutions.
Google cardboard uses a magnetic button (the button’s movement is detected by the smartphone’s magnetometer) as an input for the VR application. Bianchi & Oakley [2] have also explored the idea of using magnets on physical widgets for tablet interaction, and developed advanced physical widgets for tablet interaction. Additionally Hwang, Ahn, & Wohn [3] and Liang et al. [4] have also explored the use of magnetic accessories for tangible interaction. In this work we aim at exploring the idea of creating magnet-based attachments for VR headsets and exploring how they might be used for interaction in VR worlds.
As an example, imagine a headset attachment that provides a side rotary button on the left side and a push button on the right. In an angry birds-like VR game, the rotary button could be used to control the throwing angle and the push button could be used to trigger the throwing. In another example, a slider attachment could be used to as a timeline control in a 360º virtual reconstruction where sliding the slider would cause the system to update the visualization to a different point in time.
Objetivo
In this project the aim is to create a software toolkit for application developers with pre-defined interaction abstractions that use magnetic detection.
The objectives of this project are:
- implement the magnetic detection algorithm for smartphones proposed by [2-4]
- implement a toolkit of ready to use components for the A-Frame platform (e.g., selecting objects in the centre of view, menu navigation and selection, button activation, game controls such as fire, left, right, etc.)
- evaluate the toolkit from the perspective of VR application programmers
- use an HCI/Interaction Design approach to explore the design space of magnetic interactions
- implement one or various proof-of-concept applications that take advantage of the proposed interactions and,
- evaluate the implementation of those interactions (performance and usability of implemented interaction controls)
Plano de Trabalhos - Semestre 1
- study the literature on magnetic tangible accessories for mobile device interaction
- study the literature on programming toolkits for VR applications
- perform at least one ideation session to explore the design space of possible interactions
- implement an initial version of the toolkit with basic interactions
- Write intermediate report
Plano de Trabalhos - Semestre 2
- implement a magnetic detection solution for smartphones in the form of a software toolkit for programmers of VR applications
- evaluated the toolkit from the perspective of application programmers
- implement some of the resulting ideas into one or more VR applications
- Write the results as a scientific publication and dissertation
Condições
-
Observações
Required profile
- good programming skills, love for creating physical objects and for interaction design.
[1] Lee, J., Kim, B., Suh, B., & Koh, E. (2016). Exploring the Front Touch Interface for Virtual Reality Headsets. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’16 (pp. 2585–2591). New York, New York, USA: ACM Press. http://doi.org/10.1145/2851581.2892344
[2] Bianchi, A., & Oakley, I. (2013). Designing tangible magnetic appcessories. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction - TEI ’13 (p. 255). New York, New York, USA: ACM Press. http://doi.org/10.1145/2460625.2460667
[3] Hwang, S., Ahn, M., & Wohn, K. (2013). MagGetz: Customizable Passive Tangible Controllers on and Around Conventional Mobile Devices. In Proceedings of the 26th annual ACM symposium on User interface software and technology - UIST ’13 (pp. 411–416). http://doi.org/10.1145/2501988.2501991
[4] Liang, R.-H., Cheng, K.-Y., Chan, L., Peng, C.-X., Chen, M. Y., Liang, R.-H., … Chen, B.-Y. (2013). GaussBits: Magnetic Tangible Bits for Portable and Occlusion-Free Near-Surface Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’13 (pp. 1391–1400). http://doi.org/10.1145/2470654.2466185
Orientador
Jorge C. S. Cardoso
jorgecardoso@dei.uc.pt 📩