Print
Univis
Search
 
FAU-Logo
Techn. Fakultät Willkommen am Institut für Informatik FAU-Logo

Multi-Sensor Super-Resolution Datasets

Super-resolution algorithms reconstruct high-resolution images from low-resolution input images. For this purpose, multi-sensor super-resolution describes a technique to reconstruct high-resolution images from low-resolution data for one modality under the guidance of another modality. In our work, we investigate this concept for hybrid range imaging to super-resolve low-resolution 3-D range data that is fused with complementary photometric information (RGB data). In order to improve robustness, different computational steps of the super-resolution algorithm can be applied on the guidance data instead of using the low-resolution input images directly. This is beneficial e.g. for motion estimation required for a multi-frame super-resolution reconstruction as higher accuracy can be achieved on the guidance images.


We plan to include more datasets to consider different applications of our method in the future. The use of this data is free but please cite our corresponding paper if you would like to use it in your next publication.

 

Indoor Scene Datasets

We employed multi-sensor super-resolution to indoor range data. Our database consists of simulated data with known ground truth as well as real data acquired with real sensors (Microsoft's Kinect).

Simulated dataset. We created 4 synthetic datasets consisting of 3-D range data fused with photometric data (VGA resolution, 640 x 480 px) rendered from a an artificial scene. The data was generated using the Opens internal link in current windowRange Imaging Toolkit (RITK). Low-resolution range images are simulated by disturbing the ground truth with Gaussian blur and adding zero-mean Gaussian noise (use subsampling with desired factor to simulate reduced spatial resolution). Each dataset consists of 40 consecutive frames of the underlying scene. 

 

Organization of the data:

  • RGB (*.png): Photometric information encoded as RGB image (resolution: 640 x 480)
  • Range (*.txt): Range data with simulated Gaussian blur and zero-mean Gaussian noise  (resolution: 640 x 480). Please note that downsampling was omitted for this dataset. Use nearest-neigbour interpolation to obtain the desired pixel resolution.
  • Ground truth (*.txt): Ground truth range data  (resolution: 640 x 480).

Download the whole data set Initiates file downloadhere


Kinect datasets. We captured three datasets of indoor scenes using Microsoft's Kinect.  Range and photometric data was captured in VGA resolution (640 x 480 px) using a frame rate of 30 fps. During the acquisition, the device was held in the hand such that a small shaking ensured the required motion for super-resolution over consecutive frames.

Organization of the data:

  • RGB (*.png): Photometric information encoded as RGB image (resolution: 640 x 480)
  • Range (*.txt): Range data (resolution: 640 x 480)

Download the whole data set Initiates file downloadhere

Synthetic Endoscopy Datasets

In terms of image-guided surgery, multi-sensor super-resolution is applied in hybrid 3-D endoscopy based on Time-of-Flight (ToF) imaging. Here, we provide the datasets used for our experiments in the related publications. In the current state, the database contains synthetic endoscopic images with ground truth range data.

Baseline dataset. We created 6 synthetic datasets consisting of 3-D range data (64 x 48 px) fused with high-resolution RGB images (640 x 480 px) from a laparoscopic scene. The ground truth data was generated using the Opens internal link in current windowRange Imaging Toolkit (RITK). Low-resolution range images were simulated by disturbing the ground truth with Gaussian blur, downsample it and adding zero-mean Gaussian noise. Random movements of the virtual camera was used to simulate movements of the endoscope held by the surgeon. Small displacements of endoscopic tools and organs simulated minimally invasive surgery in a realistic manner. This images can be used as a baseline dataset, as endoscope motion simulates a small jitter of the which can be registered using optical flow and the associated 3-D data is not affected by sensor-specific errors, e.g. flying Pixels or specular reflections. Each dataset consists of 40 frames.

 

Organization of the data:

  • RGB (*.png): Photometric information encoded as RGB images
    (resolution: 640 x 480)
  • Range (*.txt): Ground truth range data and low-resolution data 
    (resolution: 64 x 48).

Download the whole data set Initiates file downloadhere

Outlier dataset. In addition to the baseline dataset, we provide 4 datasets to simulate more challenging scenarios. We simulated distance-dependent Gaussian noise, Perlin noise to simulate specular highlights in range images and flying Pixels on depth edges. In terms of motion, we considered small random endoscope movements (data set S1), larger movements of the endoscope (S2), larger tool movements (S3) and organ movements due to respiratory motion (S4). Each dataset consists of 40 frames.

 

Organization of the data:

  • RGB (*.png): Photometric information encoded as RGB images
    (resolution: 640 x 480)
  • Ground truth (*.txt) Ground truth range data (resolution: 640 x 480)
  • Range (*.txt): Simulated, noisy range data  (resolution: 640 x 480). Please note that downsampling was omitted for this dataset. Use nearest-neigbour interpolation to obtain the desired pixel resoution.

Download the whole data set Initiates file downloadhere

Software

We provide the source code of the algorithms developed for the experimental evaluations in our papers. Our Matlab super-resolution toolbox (partly accelerated using C++ and MEX) is available here.

Publications

Articles in Conference Proceedings
Köhler, Thomas; Haase, Sven; Bauer, Sebastian; Wasza, Jakob; Kilgus, Thomas; Maier-Hein, Lena; Feußner, Hubertus; Hornegger, Joachim
Bildverarbeitung für die Medizin 2014, Aachen, 2014, pp. 84-89, 2014 (BiBTeX, Who cited this?)
Köhler, Thomas; Haase, Sven; Bauer, Sebastian; Wasza, Jakob; Kilgus, Thomas; Maier-Hein, Lena; Feußner, Hubertus; Hornegger, Joachim
Medical Image Computing and Computer-Assisted Intervention - MICCAI 2013, Lecture Notes in Computer Science (International Conference on Medical Image Computing and Computer-Assisted Intervention), Nagoya, Japan, 2013, vol. 8149, pp. 139-146, 2013, ISBN 978-3-642-40810-6 (BiBTeX, Who cited this?)

Contact

For questions or comments please feel free to contact Opens window for sending emailThomas Köhler or Opens window for sending emailSven Haase