Double Your Views – Exploiting Symmetry in Transmission Imaging

Abstract

For a plane symmetric object we can find two views—mirrored at the plane of symmetry—that will yield the exact same image of that object. In consequence, having one image of a plane symmetric object and a calibrated camera, we can automatically have a second, virtual image of that object if the 3-D location of the symmetry plane is known. In this work, we show for the first time that the above concept naturally extends to transmission imaging and present an algorithm to estimate the 3-D symmetry plane from a set of projection domain images based on Grangeat’s theorem. We then exploit symmetry to generate a virtual trajectory by mirroring views at the plane of symmetry. If the plane is not perpendicular to the acquired trajectory plane, the virtual and real trajectory will be oblique. The resulting X-shaped trajectory will be data-complete, allowing for the compensation of in-plane motion using epipolar consistency. We evaluate the proposed method on a synthetic symmetric phantom and, in a proof-of-concept study, apply it to a real scan of an anthropomorphic human head phantom.

1 Introduction

Symmetry is a powerful concept with applications ranging from art to physics and mathematics [1]. This manuscript is concerned with symmetry in computer vision where we consider a theoretically sound yet surprisingly little known property of symmetric objects: when imaging a symmetric object using a calibrated camera, knowledge of the 3-D symmetry plane yields a second, virtual camera that corresponds to a mirrored version of the image seen by the true camera. This circumstance enables metric 3-D stereo reconstruction of symmetric objects using a single calibrated camera [2,3,4].

For the first time, we demonstrate that the above property naturally extends to transmission imaging, i. e. X-ray fluoroscopy, and devise image-based algorithms that exploit this circumstance to estimate intra-scan motion in circular C-arm cone-beam computed tomography (CBCT). In CBCT imaging, all camera positions are calibrated suggesting that a virtual source trajectory becomes available once the 3-D symmetry plane is known. We show that (1) this plane can be estimated efficiently from multiple projective images and (2) circular trajectories in a plane oblique to the plane of symmetry contain information that substantially benefits motion detection using recent consistency conditions.

2 Methods

2.1 Epipolar Consistency Conditions

Theory: In CBCT an X-ray source radially emits photons, that—after attenuation—are registered at a detector. The attenuation process for a ray is described by an integral. However, due to the radial structure of the rays, integrating along a detector line does not result in the Cartesian plane integral of the underlying object f but differs by a radial weighting.

Fig. 1.

Schematic drawing of a scene including two projections.

Grangeat’s theorem describes the connection between this weighted integral and a plane integral—i. e. the 3-D radon value $\mathcal {R}f(\varvec{n},d)$ describing the integral along a plane with normal $\varvec{n} \in \mathcal {S}^2$ at distance d. Using a derivative operation the radial weighting can be canceled out. Grangeat defined an intermediate function $S_\lambda (\varvec{n})$ that is calculated from projection data and is related to the derivative of the 3-D radon transform

$$\begin{aligned} S_\lambda (\varvec{n})= \int _{\mathcal {S}^2} \delta '(\varvec{x}^\top \varvec{n}) g_\lambda (\varvec{x}) d \varvec{x}= \frac{\partial }{\partial d} \mathcal {R}f(\varvec{n},d)|_{d=\varvec{c}^\top _\lambda \varvec{n}} , \end{aligned}$$

(1)

where $\delta '(\cdot )$ describes the derivative of the Dirac delta distribution, $g_\lambda (\varvec{x})$ describes a single value on the detector with $\lambda$ the projection index, $\varvec{c}_\lambda$ the source position and $\varvec{x}$ a vector from the source to a detector pixel. The geometry for two projections is visualized in Fig. 1. A detailed derivation of Eq. (1) can be found in [5], and some simplifications are discussed in [6]. From Eq. (1) it directly follows that two projections a, b must satisfy

$$\begin{aligned} S_a(\varvec{n}) = S_b(\varvec{n}) \qquad \qquad \forall \varvec{n} \in \mathcal {S}^2: \varvec{c}_b^\top \varvec{n} = \varvec{c}_a^\top \varvec{n}. \end{aligned}$$

(2)

If the geometric calibration is wrong, e. g. due to object motion, Eq. (2) will not hold. Thus, we can use it as a measure of inconsistency. The global indexing by the plane normal $\varvec{n}$ can be replaced by a local projection-pair-dependent indexing per the epipolar geometry. This particular sampling of the intermediate function is commonly denoted as epipolar consistency (EC) [6] and allows to efficiently evaluate redundant values only based on the corresponding projection matrices $\varvec{P}_a$ and $\varvec{P}_b$, allowing the reformulation of Eq. (2) to

$$\begin{aligned} \varvec{S}_a(\varvec{P}_a, \varvec{P}_b) = \varvec{S}_b(\varvec{P}_b, \varvec{P}_a) , \end{aligned}$$

(3)

where $\varvec{S}$ denotes the array of intermediate values computed from S.

Short Scans and the Circular Trajectory: State of the art head imaging protocols for CBCT consist of a circular trajectory of 496 projections. Hereby the source detector gantry rotates around the patient covering a $200^\circ$ segment. All source positions within the trajectory lie on a plane, typically referred to as trajectory plane. We define this plane coincident with the $\varvec{x},\varvec{y}$-plane.

Rigid patient motion or geometry misalignment can be estimated and compensated for using EC [7,8,9]. This is achieved by finding a rigid transformation $\varvec{T}_i$ for each projection matrix $\varvec{P}_i$, accounting for the patient motion at acquisition of projection i. The motion is expected to be compensated, when the inconsistency between all projections is minimal. The result of motion estimation is a set of rigid transformations $\varvec{T}= [\varvec{T}_1,\dots \varvec{T}_{496}]$ that satisfy

$$\begin{aligned} \hat{\varvec{T}} = \mathop {\text {arg min}}\limits _{\varvec{T}} \sum _{a,b = 1}^{496} \left||\varvec{S}_a(\varvec{P}_a\varvec{T}_a,\varvec{P}_b\varvec{T}_b) - \varvec{S}_b(\varvec{P}_b\varvec{T}_b,\varvec{P}_a\varvec{T}_a)\right||_2. \end{aligned}$$

(4)

However, there are theoretical limitations due to the geometry of the circular trajectory. Most radon planes of the object that include two source positions are almost parallel to the trajectory plane. As the consistency is based on the radon value, only motion that steps out of the radon plane is detectable. As a consequence only out-plane motion ($r_x, r_y, t_z$) can be estimated well, while in-plane motion ($t_x, t_y, r_z$) cannot be estimated robustly [7]. In the following, we show that we can exploit symmetry to generate a short scan-like trajectory which is data complete and, thus, beneficial for estimation of in-plane motion.

2.2 Symmetric View Augmentation

Symmetry in Transmission Imaging: Consider a plane-symmetric scene as depicted in Fig. 2. The two points $\varvec{x}_a\in \mathbb {P}^3$ and $\varvec{x}_b\in \mathbb {P}^3$ are symmetric to the $\varvec{z},\varvec{y}$-plane. This bilateral symmetry relation can be expressed by an involutive isometric transformation $\varvec{F}$—i. e. a reflection matrix—as

$$\begin{aligned} \varvec{x}_a= \varvec{F}\, \varvec{x}_b\qquad \qquad \varvec{x}_b= \varvec{F}\, \varvec{x}_a, \end{aligned}$$

(5)

where $\varvec{F}\in \mathbb {R}^{4\times 4}$ only flips the sign of the x component. Since $\varvec{F}$ is an isometric transformation we find the mirrored projection matrix as $\varvec{P}\varvec{F}$ (cf. Fig. 2). The resulting image on the right detector will be the projection of the points $\varvec{x}_a$ and $\varvec{x}_b$ under $\varvec{P}$ and the resulting image on the left detector will be the projected points $\varvec{x}_a$ and $\varvec{x}_b$ under $\varvec{P}\varvec{F}$ giving

$$\begin{aligned} \varvec{u}_a = \varvec{P} \varvec{x}_a\qquad \quad \varvec{u}_b = \varvec{P} \varvec{x}_b\qquad \quad \varvec{u}_a^\prime = \varvec{P}\varvec{F}\varvec{x}_a\qquad \quad \varvec{u}_b^\prime = \varvec{P}\varvec{F}\varvec{x}_b. \end{aligned}$$

(6)

Inserting the symmetry relation given by Eq. (5) in the two leftmost equations of Eq. 6 gives the relation

$$\begin{aligned} \varvec{u}_a = \varvec{P} \varvec{x}_a= \varvec{P}\varvec{F}\varvec{x}_b= \varvec{u}_b^\prime \qquad \qquad \varvec{u}_b = \varvec{P} \varvec{x}_b= \varvec{P}\varvec{F}\varvec{x}_a= \varvec{u}_a^\prime . \end{aligned}$$

(7)

This result allows to conclude that both detector images will exhibit the exact same image. Note that this only holds since the reflection of the projection matrix also flips the $\varvec{u}$ and $\varvec{v}$ axis. Consequently, a transmission image of a plane symmetric object can be interpreted as acquired under either the projection $\varvec{P}$ or $\varvec{P}\varvec{F}$. This observation allows to effectively double the views of an acquisition if the symmetry plane is known.

Fig. 2.

Visualization of a plane symmetric scene.

Symmetry Plane Estimation: We apply EC (cf. Eq. (4)) to find the reflection $\varvec{F}^\prime$ that represents the most consistent transformation, which is—by definition—the reflection at the symmetry plane. To optimize for a certain symmetry plane, we need to find the transformation describing $\varvec{F}^\prime$, which is given by $\varvec{F}^\prime = \varvec{T}\varvec{F}\varvec{T}^{-1}$, with $\varvec{T}$ being a rigid transformation. The reflection $\varvec{F}^\prime$ is then found by optimizing for $\hat{\varvec{\varvec{T}}}$ minimizing the inconsistency defined as

(8)

where N is the number of projections used for finding the symmetry plane.

Fig. 3.

Visualization of X-trajectory. The acquired trajectory is embedded in the blue trajectory plane (blank dots) and the mirrored virtual trajectory is embedded in the red trajectory plane (solid dots).

The X-Trajectory: If the symmetry plane of the scanned object is oblique to the trajectory plane of a short scan by an angle $\alpha$ as visualized in Fig. 3, the mirrored trajectory plane will be rotated to the acquired trajectory plane by $2\alpha$. Thus, for adequate angles $\alpha$, the combined trajectory fulfills Tuy’s condition and the short scan becomes data complete. This in turn enables the use of Grangeat’s theorem to detect in-plane motion.

Fig. 4.

(a): Slice through the symmetric phantom. (b): Digitally rendered radiographs (DRR) of the phantom. (c): Aligned reconstruction of anthropomorphic head phantom. (d): raw projection data from a short scan of the head phantom.

2.3 Experiments

Data: To evaluate our method we synthetically generated a plane symmetric phantom, consisting of four small balls, and two half spheres (cf. Fig. 4a). From this phantom, a short scan is simulated in silico (cf. Fig. 4b). The second dataset is a short scan acquired from a real anthropomorphic human head phantom using a robotic C-arm system (Artis zeego, Siemens Healthcare GmbH, Germany). The phantom is placed, such that the expected symmetry plane is oblique to the trajectory plane. As expected in a real clinical case, the head phantom does not exhibit a perfect plane symmetry. A slice through the reconstruction and projections from the acquired short scan of the head are shown in Figs. 4c and 4d, respectively.

Estimation of Symmetry Plane: Using the synthetic and real datasets, we first estimate the symmetry plane. The plane is found from projection domain images only by minimizing Eq. (8) using the Nelder-Mead method. The optimization searches for the symmetry plane parameters described by three DoF.

Application to Rigid Motion: To study the impact of the X-trajectory in dependence of $\alpha$ on in- and out-plane motion, we add a rigid spline motion to each motion parameter. The motion amplitude is in the range of ±0.3 mm or degree, respectively, and distributed only in the central part of the trajectory, where no opposing views are available. Then we compute combined consistency grids of the motion affected trajectory using the synthetic phantom. The grid is build up by a ($N\,{\times }\,N$) matrix $\varvec{C}$, where each element $c_{ab}$ denotes the consistency between views a and b. The lower-left triangle of the consistency grid denotes the conventional EC (CEC) computed from two views on the circular trajectory. The upper-right triangle is computed as the EC between an acquired view a and a mirrored view b, which we denote as mirrored EC (MEC).

In addition we inspect the inconsistency induced by an $r_z$ motion pulse distributed over view 238–288 on the acquired human head phantom consisting of 496 views. We compare the CEC solely computed from the short scan, epipolar consistency between mirrored and acquired view (MEC) and a combination of both (combined CEC and MEC).

3 Results and Discussion

Estimation of Symmetry Plane: The estimated and ground truth symmetry plane parameters for the synthetic phantom are listed in Table 1. Estimation succeeded with very high accuracy.

Fig. 5.

Reconstruction of acquired head phantom. The volume is aligned to the symmetry plane (white line) and shown from an axial and coronal view.

Table 1. Ground truth symmetry plane parameters (normal and signed distance from origin) of synthetic phantom and estimated symmetry plane parameters.

The head phantom, while not perfectly symmetric, exhibits a well defined symmetry plane that was estimated very robustly. A reconstruction aligned w.r.t. the symmetry plane is depicted in Fig. 5.

Application to Rigid Motion: Comparing upper and lower row of Fig. 6 that correspond to $\alpha = 0^\circ$ and $\alpha =30^\circ$, respectively, the impact of the X-trajectory is evident. The CEC (lower left triangle of the grid) detects inconsistency within out-plane parameters (three rightmost columns) while in-plane motion (three leftmost columns) is not detected well. Using MEC and an angle $2\alpha =60^\circ$ between the acquired and mirrored trajectory plane, prominently reveals in-plane motion.

Fig. 6.

Inconsistency due to motion in the trajectory using the synthetic phantom. Bright pixels encode a high inconsistency and dark regions encode consistent view pairs. Upper row: $\alpha = 0^\circ$, lower row: $\alpha =30^\circ$.

Fig. 7.

Sensitivity of consistency measure to motion impulse. Left column: CEC. Middle column: MEC. Right column: combined CEC and MEC.

Figure 7 shows the different consistency measures (CEC, MEC, combined CEC and MEC) responding to a motion impulse on the acquired data. All measures are able to detect large scale motion. However, using CEC the global optimum is displaced by $0.2^\circ$. The MEC optimum is displaced by $0.03^\circ$, combined CEC and MEC shows a displacement of $0.06^\circ$, suggesting that, in this setup, MEC is more accurate than CEC by a factor of 10.

4 Conclusion

We presented the concept of plane symmetry for transmission imaging and provided an algorithm to estimate the 3-D plane of symmetry based on projection images only. In combination with a short scan trajectory oblique to the symmetry plane, an X-shaped trajectory arises that is associated with several benefits. For adequate angles $\alpha$, both in- and out-of-plane motion directions are detectable using Grangeat’s theorem. This property naturally arises from the observation that the X-trajectory is Tuy-complete. We have evaluated the proposed algorithm on a real scan of an anthropomorphic head phantom. Despite being only partially symmetric, the proposed concept of exploiting symmetry was still found applicable. Future research is needed to find effective optimization strategies to estimate complex motion patterns. We conclude that symmetry is a powerful concept in transmission imaging with the potential to benefit diverse imaging problems that make use of consistency condition such as calibration, beam hardening- and truncation-correction.

Disclaimer: The concepts and information presented in this paper are based on research and are not commercially available.