Techn. Fakultät Willkommen am Institut für Informatik FAU-Logo

Peter Fischer M. Sc.

Alumnus of the Pattern Recognition Lab of the Friedrich-Alexander-Universität Erlangen-Nürnberg

Respiratory Motion Compensation in Interventional Fluoroscopy using Computer Vision and Machine Learning
Motion Compensation for Overlays in Interventional Fluoroscopy

X-ray fluoroscopy guidance using a C-arm CT is widely used for minimally invasive interventions on the heart or the liver. However, soft tissue contrast in fluoroscopic images is low. Consequently, preoperatively acquired roadmap information, e.g. from MR or CT, is overlaid onto the fluoroscopic images during the intervention. In the thorax or abdomen, the respiratory motion of the patient causes inconsistencies between the live fluoroscopic images and a static roadmap overlay.

This project researches respiratory motion compensation. This has the potential to drastically increase the reliability and accuracy of overlays. A model-based approach is investigated, which uses an externally measured or image-based respiratory signal to control a respiratory model. A challenge is the reliable estimation of respiratory information from the fluoroscopic images. Algorithms for the extraction of a respiratory signal from the images are developed and analyzed. In addition, several methods for measuring the respiratory signal externally are investigated and compared. Another key aspect is the analysis and development of respiratory models. Preoperative measurements can be used to create the respiratory model.

Respiratory Signal Extraction From X-Ray Fluoroscopy Sequences
P. Fischer, T. Pohl, A. Maier, J. Hornegger
  • X-ray image used for respiratory signal extraction.

    In X-ray fluoroscopy-guided minimally invasive interventions, overlays of pre-procedurally acquired image data can be used to visualize soft-tissue. In the thoracic and abdominal regions, static overlays are inconsistent to the live X-ray images due to respiratory motion of the patient. This error can be reduced by dynamically adapting the overlay to the respiration. A first step in this direction is the real-time extraction of the respiratory state from the live X-ray images. The respiratory state can drive a motion model to compensate the breathing motion.

    We present a method to extract respiratory signals from X-ray sequences in real-time. Respiratory signal extraction is viewed as a dimensionality reduction problem, which is performed for each X-ray image using incremental dimensionality reduction. This approach is applicable for many X-ray angles, robust to noise, and can deal with complex motion patterns. In addition, linear (PCA) and nonlinear (Manifold Learning) dimensionality reduction techniques were compared. Both techniques can reach real-time performance and a high accuracy. Manifold learning is more accurate but slower.

    Articles in Conference Proceedings
    2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI) (International Symposium on Biomedical Imaging (ISBI)), Beijing, China, 29.04.2014, pp. 915-918, 2014, ISBN 978-1-4673-1961-4 (BiBTeX, Who cited this?)
    IGIC 2014 - Abstractband (Conference on Image-Guided Interventions), Magdeburg, 14.10.2014, pp. 80-81, 2014 (BiBTeX, Who cited this?)
Motion Estimation and Layer Separation in X-ray Fluoroscopy
P. Fischer, T. Pohl, T. Köhler, A. Maier, J. Hornegger
  • Layers extracted from an X-ray image sequence.
    X-ray layers

    Motion estimation in X-ray fluoroscopy images is challenging due to the nature of image generation. X-rays are attenuated as they pass through the imaged volume, resulting in transparently overlapping structures from different depths in 2-D. The motion in the 2-D images cannot be explained by a 2-D transformation. Consequently, conventional image registration algorithms are not applicable for motion estimation.

    One feasible approach is tracking of high-contrast structures, e.g. catheters, bones. We compared different feature detectors and descriptors for this purpose in "On Feature Tracking in X-ray Images". This leads to sparsely distributed motion estimates.

    For dense motion estimation, we investigated layered approaches to tackle the transparency problem. The transparent projection from 3-D to 2-D is approximated by a summation of multiple 2-D layers. The transformations in the X-ray images can be explained better by this model, but the registration problem is much more difficult. We made several contributions in this area.

    • In "Markov Random Field-based Layer Separation for Simulated X-Ray Image Sequences", we separate the layers without knowledge of the layer motions using a MRF model.
    • We propose a robust probabilistic model for layer separation if the motion of all the layers are known. This is a substep in joint layer and motion estimation.
    • We introduce surrogate signals as prior information for the motion of each layer. This enables the estimation of dense, globally consistent, physiologically plausible motion from X-ray image sequences for the first time.
    Articles in Conference Proceedings
    Bildverarbeitung für die Medizin 2015 (Bildverarbeitung für die Medizin), Lübeck, 17.03.2015, pp. 329-324, 2015 (BiBTeX, Who cited this?)
    24th International Conference on Information Processing in Medical Imaging, IPMI 2015 (Information Processing in Medical Imaging), Isle of Skye, Scotland, 28.06.2015, pp. 288-299, 2015, ISBN 978-3-319-19991-7 (BiBTeX, Who cited this?)
    Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 (International Conference on Medical Image Computing and Computer-Assisted Intervention), München, 06.10.2015, pp. 282-289, 2015, ISBN 978-3-319-24552-2 (BiBTeX, Who cited this?)
    Bildverarbeitung für die Medizin 2014, Aachen, 18.03.2014, pp. 132-137, 2014 (BiBTeX, Who cited this?)