Titulo Estágio
Implicit 3D Thermography
Áreas de especialidade
Sistemas Inteligentes
Local do Estágio
ISR-Polo2
Enquadramento
Infrared (thermographic) images suffer from low resolution and a lack of distinguishable features to
perform 3D reconstruction algorithms such as structure from motion, Gaussian splatting, …
Some solutions have been proposed:
Martin Landmann, Stefan Heist, Patrick Dietrich, Peter Lutzke, Ingo Gebhart, Joachim Templin, Peter Kühmstedt, Andreas Tünnermann, Gunther Notni, High-speed 3D thermography, Optics and Lasers
in Engineering, Volume 121, 2019, Pages 448-455, ISSN 0143-8166,
https://doi.org/10.1016/j.optlaseng.2019.05.009.
Sayed Pedram Haeri Boroujeni, Abolfazl Razi, IC-GAN: An Improved Conditional Generative Adversarial Network for RGB-to-IR image translation with applications to forest fire monitoring,
Expert Systems with Applications, Volume 238, Part D, 2024, 121962, ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2023.121962.
We want to explore another idea: implicit stereo calibration (as opposed to explicit camera calibration). We take two infrared cameras and take images of checkerboards. From this information, we calibrate the stereo system. This calibration gives us the 3D information. This
method is an explicit geometric method (we also calibrate each camera explicitly in the process).
Alternatively, we want to use even more cameras. We now calibrate the whole system instead of each individual camera. To do this, we find correspondences between the camera images. We move a ball around with a robot. The ball can be heated up, so that is shows in the thermal images. The robot tracks the 3D coordinates of the ball. After we take some images, we have a set of correspondences between the images and the 3D coordinates. We train a machine learning algorithm on these correspondences. Given a new point in the images, we can now predict the 3D coordinate in the real world.
Objetivo
We want to explore another idea: implicit stereo calibration (as opposed to explicit camera
calibration). We take two infrared cameras and take images of checkerboards. From this
information, we calibrate the stereo system. This calibration gives us the 3D information. This
method is an explicit geometric method (we also calibrate each camera explicitly in the process).
Alternatively, we want to use even more cameras. We now calibrate the whole system instead of
each individual camera. To do this, we find correspondences between the camera images. We move a ball around with a robot. The ball can be heated up, so that is shows in the thermal images. The
robot tracks the 3D coordinates of the ball. After we take some images, we have a set of
correspondences between the images and the 3D coordinates. We train a machine learning algorithm on these correspondences. Given a new point in the images, we can now predict the 3D
coordinate in the real world.
Plano de Trabalhos - Semestre 1
The goal of this project is to implement the idea of both explicit and implicit stereo (or more than 2)
camera calibration for infrared cameras so that we can perform 3D thermography.
Objectives:
1. Make a ball move around in the lab using a robot. Th ball has to be moved around in a
defined 3D volume.
2. Make the robot log the 3D coordinates.
3. Take images with the cameras. These have to be in sync with each other, and also in sync
with the positions the robot tracks.
4. Compose a dataset of pixel coordinates to 3D coordinates. Use this to train a machine
learning algorithm.
Plano de Trabalhos - Semestre 2
5. Validate on 3D object with known geometry. Does the predicted 3D model match the real-
world model?
6. Map the temperatures of the thermal camera images to the 3D model.
7. Compare this to stereo calibration with checkerboards
Condições
Desenvolvimento do trabalho no ISR em Coimbra e na Universidade de Antuérpia.
Observações
Sem elementos adicionais.
Orientador
Helder Araújo
helder@isr.uc.pt 📩