In X-ray fluoroscopy-guided minimally invasive interventions, overlays of pre-procedurally acquired image data can be used to visualize soft-tissue. In the thoracic and abdominal regions, static overlays are inconsistent to the live X-ray images due to respiratory motion of the patient. This error can be reduced by dynamically adapting the overlay to the respiration. A first step in this direction is the real-time extraction of the respiratory state from the live X-ray images. The respiratory state can drive a motion model to compensate the breathing motion.

We present a method to extract respiratory signals from X-ray sequences in real-time. Respiratory signal extraction is viewed as a dimensionality reduction problem, which is performed for each X-ray image using incremental dimensionality reduction. This approach is applicable for many X-ray angles, robust to noise, and can deal with complex motion patterns. In addition, linear (PCA) and nonlinear (Manifold Learning) dimensionality reduction techniques were compared. Both techniques can reach real-time performance and a high accuracy. Manifold learning is more accurate but slower.

Motion estimation in X-ray fluoroscopy images is challenging due to the nature of image generation. X-rays are attenuated as they pass through the imaged volume, resulting in transparently overlapping structures from different depths in 2-D. The motion in the 2-D images cannot be explained by a 2-D transformation. Consequently, conventional image registration algorithms are not applicable for motion estimation.

One feasible approach is tracking of high-contrast structures, e.g. catheters, bones. We compared different feature detectors and descriptors for this purpose in "On Feature Tracking in X-ray Images". This leads to sparsely distributed motion estimates.

For dense motion estimation, we investigated layered approaches to tackle the transparency problem. The transparent projection from 3-D to 2-D is approximated by a summation of multiple 2-D layers. The transformations in the X-ray images can be explained better by this model, but the registration problem is much more difficult. We made several contributions in this area.

• In "Markov Random Field-based Layer Separation for Simulated X-Ray Image Sequences", we separate the layers without knowledge of the layer motions using a MRF model.
• We propose a robust probabilistic model for layer separation if the motion of all the layers are known. This is a substep in joint layer and motion estimation.
• We introduce surrogate signals as prior information for the motion of each layer. This enables the estimation of dense, globally consistent, physiologically plausible motion from X-ray image sequences for the first time.