Keywords

1 Introduction

In clinic practice, CT scan is one of the most popular methods to obtain digital images of human livers. In the last few decades, computer scientists have made great efforts to separate the liver from other organs in the abdominal CT. For example, the Grand Challenge of MICCAI 2007 calls for many liver segmentation methods, the summary of which can be found in a 2009 publication of Heimann et al. These methods could be divided into three categories, including gray level based method, shape based method and pattern based method [14].

The intensity of CT image is used for segmentation in the gray level based method. Several classic algorithms like region growing, level set and graph cut can fit into this category. The region growing method, first introduced by Pohle et al. [19], is widely employed to segment brain tissues [1], thoracic aorta [15] and abdominal organs [23]. In MICCAI 2007, both Rusko et al. [20] and Beck and Aurich [2] employ a 3D region growing tool to segment the liver. This method is further improved by Yuan et al. [24] and Elomorsy [8] through adding pre- and post-processing steps to handle the leakage problem of region growing method. It is also used as an initialization step for more complex pipelines presented by Goryawala et al. [9] and Chu et al. [5]. In this paper, we also use the region growing method to find the rough area of the liver.

The two most frequently used models for the shape based method are statistical shape model (SSM) [6] and probabilistic atlases (PA) [10, 17]. However, the newly developed Sparse Shape Composition (SSC) model challenges these two methods by claiming that there is not an accurate “mean shape” for all livers [22]. It decomposes the liver to multiple regions and thus increases the flexibility of the shape by taking account of shape within each region.

The multiple texture features of the liver in CT are taken into account in the pattern based method. For example, Christ et al. [4] and Ben-Cohen et al. [3] all train a fully convolutional neural network (FCNN) to achieve the semantic liver segmentation from the CT images, while Danciu et al. [7], Luo et al. [13] and Zhang et al. [25] use SVM to train the classifier.

In this study, we take a different approach by taking the internal structure inside the liver as the reference for liver segmentation. Specifically, we segment vessel system in the liver, then use it to identify the component of liver. In our algorithm, we choose the hepatic vein system from the three tube systems inside the liver [21], because it displays the most obvious intensity contrast against cellular tissue of the liver. We also choose the CT images obtained at portal venous phase during which the agent highlights the hepatic veins most significantly.

In brief, we (1) segment hepatic veins from abdominal CT; (2) use the tree-structure of vein to define which mess around them belongs to the liver; (3) apply isolated point cleaning algorithm to selected liver voxels to further define liver boundary; (4) compare resultant liver boundary to a gold standard, i.e. the manual defined liver boundary by an experimenter on each slide of CT images.

2 Method

In our approach, we construct a probabilistic model to segment liver. There are four probabilities corresponding to four different features of the liver voxel in our probabilistic model, including the intensity, location, distance to hepatic veins and connectivity of the candidate voxels. The “probability” here is a generalized name for the four measurements we use for these features. It could be binary or continuous function.

2.1 \(P_1\): Liver Intensity Range Probability

\(P_1\) is to search for voxels in a certain intensity range defined by \(\mu _l\) (center) and \(\rho _l\) (range). This idea is valid because the liver occupies a continuous and relatively concentrated area in the histogram of abdominal CT. To define the intensity range of liver, we set a threshold manually according to the real-time visual feedback of our interface. When the voxel intensity \(I(\varvec{x})\) is beyond this threshold, the value of \(P_1\) will be set to 0 directly. When \(I(\varvec{x})\) falls inside the pre-defined intensity threshold, the \(P_1\) will be calculated according to their similarity to the center of the threshold (Formula (1)).

$$\begin{aligned} P_1(\varvec{x})=\left\{ \begin{array}{lcl} 1-\frac{|I(\varvec{x}-\mu _l)|}{\rho _l} &{}&{} {|I(\varvec{x})-\mu _l|\le \rho _l}\\ 0 &{}&{} {|I(\varvec{x})-\mu _l|>\rho _l} \end{array} \right. \end{aligned}$$
(1)

Before applying the intensity-based filter, we are also aware of noises in the original CT images. So we apply the anisotropic diffusion filter (Perona-Malik [18]) to remove small gratitude change caused by noise as well as keep the large gratitude change caused by intensity difference near liver edge. Figure 1 shows the improved outcome of \(P_1\) when the images are smoothed by the anisotropic diffusion filter first. Noises inside the liver are significantly removed (Fig. 1(d)).

Fig. 1.
figure 1

Noises in \(P_1\) result are significantly eliminated after the anisotropic diffusion filter is applied.

Full size image

2.2 \(P_2\): Liver Location Probability

The liver, for the majority of human, is located at the same position of the body [12]. This feature is used to define location probability \(P_2\). The goal of \(P_2\) is to produce a rough area of liver. We employ the region growing algorithm and morphological operations to achieve this goal.

In particular, we put a seed inside the liver on an arbitrary slice. Then we apply the region growing algorithm with a small similarity threshold to expand the region from the seed point. This is to prevent the region from growing to other neighboring organs. As a result, holes appear as shown in Fig. 2(b). So we further apply the morphological dilation to fill in the holes (Fig. 2(c)). The radius of the dilation is kept small to better preserve the outline of the liver. Small radius of the dilation leaves some holes not completely filled. So we finally apply the morphological closing and all of the holes can be perfectly filled (Fig. 2(d)).

Fig. 2.
figure 2

Region growing algorithm and morphological operations are employed to get a rough area of the liver.

Full size image

2.3 \(P_3\): Hepatic Vein Neighborhood Probability

The hepatic veins inside the liver are ideal reference for defining the boundaries of the liver. The hepatic cells need to be nourished by blood from haptic veins that reach every corner of the liver. Meanwhile, the hepatic veins could only supply blood to cells located within a certain distance to them. This is the anatomical foundation for \(P_3\).

For vessel extraction, we apply a threshold filter within the region of \(P_2\) instead of applying the region growing algorithm to the whole volume. Because the former is not affected by the connectivity of the vessel structure in CT images and is able to detect all the candidate vessel points within the reference region (Fig. 2). To formulate this method, we define the hepatic vein possibility \(P_{3v1}\). For the voxel with intensity \(I(\varvec{x})\) out of the region of \(P_2\) or the range \((\mu _v\pm \rho _v)\), we set \(P_{3v1}\) to 0. And for voxels that are within the region of \(P_2\) and the range \((\mu _v\pm \rho _v)\), we calculate the \(P_{3v1}\) according to their intensity similarity to a predefined center \(\mu _v\).

$$\begin{aligned} P_{3v1}(\varvec{x})=\left\{ \begin{array}{lcl} 1-\frac{|I(\varvec{x}-\mu _v)|}{\rho _v} &{}&{} {|I(\varvec{x})-\mu _v|\le \rho _v\ \wedge \ P_2(\varvec{x})>0}\\ 0 &{}&{} {|I(\varvec{x})-\mu _v|>\rho _v\ \vee \ P_2(\varvec{x})\le 0} \end{array} \right. \end{aligned}$$
(2)

Wide coverage of the second method also brings new problems. The most critical one is that those isolated points with similar intensity to hepatic veins may be faultily selected, which decreases the accuracy of \(P_3\) and further reduces the efficiency of the algorithm. We develop an algorithm to prevent these isolated points from being wrongly added to the point cloud of hepatic veins. In this algorithm, the 26-neighbourhood of every candidate vessel points is checked and only those that have a certain number of vessel points in their neighborhood could be preserved. A more detailed description of this process is shown in Algorithm 1.

Fig. 3.
figure 3

The coverage of threshold based vessel extraction method (red area in (c)) is wider than that of region growing based method (blue area in (b)). (Color figure online).

Full size image
figure a

For the hepatic vein neighborhood probability \(P_3\), we calculate it based on the point cloud of hepatic veins. We find the n-nearest neighbors of each voxel from the point cloud and record the distance between each pair. Then the probability of current voxel \(\varvec{x}\) as a near neighbor of hepatic veins can be measured by the ratio of the hepatic vein possibility \(P_{3v1}(\varvec{x})\) and corresponding distance \(distance(\varvec{x},\varvec{x_i})\) between \(\varvec{x}\) and \(\varvec{x_i}\). As we consider n neighbors of each voxel, the final possibility is an average of n ratios generated by previously described method. Algorithm 2 shows details of this process.

$$\begin{aligned} P_3(\varvec{x})=\varPhi (P_{3v1}(\varvec{x}),d)\sim \frac{1}{n}\sum _{\varvec{x}\in {N}}\frac{P_{3v1}(\varvec{x_i})}{distance(\varvec{x},\varvec{x_i})} \end{aligned}$$
(3)

The running on \(P_3\) for the whole volume is a time-consuming process. So we construct a k-d tree for the point cloud of hepatic veins to reduce the algorithm complexity. In this way it turns the time complexity of n-nearest neighbor searching from O(n) to O(logn). And we just run \(P_3\) for voxels with non-zero \(P_1\) value, because it is a necessary but not sufficient condition for the liver voxel.

In brief, there are two key steps for running the hepatic vein neighborhood probability \(P_3\): (1) choose candidate hepatic vein points and assign hepatic vein probability \(P_{3v1}\); (2) calculate the final \(P_3\) value based on \(P_{3v1}\) and \(P_2\). Algorithm 2 shows details of this process.

figure b

2.4 \(P_4\): Liver Voxel Neighborhood Probability

Since the human liver is a solid and interconnected entity, the goal of \(P_4\) is to punish voxels that are not close to other liver voxels. Strictly speaking, we are not calculating a new probability. We just apply the cleansing algorithm previously used for \(P_3\) to further remove isolated points.

We first construct a point cloud with all voxels with their \(P_3\) bigger than a predefined lower bound \(\rho _{low}\). Then we apply the cleansing algorithm (Algorithm 1) to find isolated points inside the point cloud. Finally, we get the value of \(P_4\) for each voxel according to Formula (4). For isolated voxels, we don’t remove it directly but subtract a constant \(\kappa \) from its \(P_3\) value.

$$\begin{aligned} P_4(\varvec{x})=\left\{ \begin{array}{lcl} P_{3}(\varvec{x})-\kappa &{}&{} {P_{3}(\varvec{x})\ge \rho _{low} \wedge \ \varvec{x}\ is\ isolated}\\ P_{3}(\varvec{x}) &{}&{} {P_{3}(\varvec{x})\ge \rho _{low} \wedge \ \varvec{x}\ is\ not\ isolated}\\ 0 &{}&{} {P_{3}(\varvec{x})<\rho _{low}} \end{array} \right. \end{aligned}$$
(4)

2.5 Final Probability

The final probability is calculated for every voxel as the product of \(P_1\) and \(P_4\). If the probability is larger than a predefined constant \(\eta \), the corresponding voxel is considered as a liver voxel and gets the mask value true. Otherwise, it gets the mask value false. The output of the final result is a binary image which stores the region of segmented liver.

3 Experiment and Results

3.1 Experimental Setup and Evaluation Measures

We set up a control experiment to evaluate the accuracy of our algorithms for liver segmentation. The experiment environment was built on the open source medical image processing platform 3DMed developed by the Chinese Academy of Science [16]. We develop a plugin in it to test our algorithm interactively.

The evaluation is measured by variables used in Grand Challenge of MACCAI 2007. We compare the segmentation outcomes with expert-generated reference and rate them according to five measures: Volumetric Overlap Error, Relative Volume Difference, Average Symmetric Surface Distance (ASD), Root Mean Square Symmetric Surface Distance (RMSD) and Maximum Symmetric Surface Distance (Hausdorff distance) (MSD) [11]. Besides, we add the accuracy measure as shown in Formula (5). It is to show the difference between the algorithm segmented result A and the manual segmented result B more clearly.

$$\begin{aligned} Accuracy=100(|A\cap {B}|/|B|) \end{aligned}$$
(5)

3.2 Results

Vessel Extraction. The accuracy of vessel extraction is important because it serves as the foundation for other algorithms in this pipeline. Our algorithm extract liver vessels based on the intensity and location. We compared our vessel extraction outcomes to the result from the dynamic region growing tool provided by Mimics 17.0 on the same dataset. Figure 4 shows that our outcome (Fig. 4 (a)) is more accurate than the Mimics output (Fig. 4(b)) in terms of the ability to eliminate unnecessary part connected to hepatic veins. And in our method, much thinner branch of the hepatic veins could be identified and displayed to surgeons (Fig. 4(a)).

Fig. 4.
figure 4

The vessel structure extracted by our method (a) is better than that extracted by the dynamic region growing tool provided in Mimics 17.0.

Full size image

Liver Segmentation. In fact, we produce two segmentation results in our pipeline for each dataset. The first one is the rough liver area reported simply by the location probability \(P_2\). We call it “location based method”. The second one is the final segmentation result jointly determined by four probabilities mentioned above. We call it “vessel distance based method”. Segmentation outcomes of these two models were then compared to default outcome of Mimics 17.0 (Table 1).

Table 1. A Comparison of segmentation result
Full size table

The location based method produces better segmentation result than the vessel distance based method, and both of them produce better segmentation result than Mimics 17.0. The outcome of location based method shows significantly higher accuracy rate, lower overlap error and surface distance compared to the Mimics output. The vessel distance based method is not that superior, but it still produces higher accuracy as well as lower overlap error, volume difference (absolute value), RMS distance and Max. Distance than the Mimics output. We notice that the vessel distance based method can achieve shortest RMS distance compared to the location based method and Mimics. This indicates that the segmentation result of vessel distance based method is good at finding all the candidate voxels of the liver, but it may introduce mistakes in a more detailed view.

Result Comparison. Four sets of CT images are obtained and manually segmented by a human expert. The expert’s segmentation serves as the ground truth. Outputs from Mimics, location based and vessel distance based algorithms are compared to the ground truth in terms of 3D volume (Fig. 5).

Fig. 5.
figure 5

The first column is \(P_2\) Result by location based method. The second row is final result by vessel distance based method. The thrid column is the Mimics result by the tools from Mimics 17.0. The 3D visualization of the segmentation results shows that the location based method produces the most accurate result among the three segmentation strategies.

Full size image

By inspecting results in Fig. 5, we can find that the location based method has the ability to eliminate other tissues that are connected to or have similar intensity to the liver. The vessel distance based method, however, shows lower capacity to remove those neighboring points whose intensity lies in the same threshold as liver. The result produced by the “dynamic region growing” tool of Mimics presents an unstable capacity to segment the liver. For some part of the liver, Mimics is able to produce clear and smooth boundary that is comparable to the location based method. But for other parts, leaking to other neighboring organs such as the spleen and kidney is likely.

The outcome of location based method and vessel based method show better performance than Mimics 17.0. The location based method has the ability to eliminate other tissues that are connected to or have similar intensity to the liver. The vessel distance based method can achieve shortest RMS distance compared to the location based method and Mimics. However, the vessel distance based method shows lower capacity to remove those neighboring points whose intensity lies in the same threshold as liver. The result produced by the “dynamic region growing” tool of Mimics presents an unstable capacity to segment the liver.

3.3 Discussion

We believe that the major reason for the unsatisfactory results of the vessel distance based liver segmentation approach is our limited ability to perform an optimal vessel extraction from current CT images. In the cases we study, the cross sectional scanning taken by CT is at 0.5–1 mm horizontally and 1.3–5 mm vertically, which is not precise enough to show small blood vessel like capillary. Capillaries are the kind of vessel we expect to use for our distance based method. Unfortunately, it is not available with current CT images. We plan to apply this algorithm to higher-resolution CT images where distance between each cross-sectional scanning can be as short as 0.2 mm, so that it is capable of identifying thinner vessels and producing better result.

4 Conclusion

In this paper we propose an interior structure based liver segmentation strategy on CT images. The main idea is to extract the hepatic vein structure and then use it as a reference to find the boundary of the liver. Our pipeline considers the distance to hepatic veins, and three other features including intensity, location and connectivity of the liver. We integrate them into a probabilistic model. We test our segmentation algorithm on real clinical data and the result is better than the output from Mimics. However, it is not superior to location based method when we perform the test with conventional CT scan. We hope our algorithm will yield a better outcome when applied to the high-resolution data.